[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:46 dist-test-slave-cxpd krb5kdc[20950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:46 dist-test-slave-cxpd krb5kdc[20950](info): set up 2 sockets
Sep 24 21:16:46 dist-test-slave-cxpd krb5kdc[20950](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260924 21:16:47.925275 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:16:48.044306 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:37209
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:37209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:48.126572 20963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:48.126777 20963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:48.126801 20963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:48.127945 20963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:48.127986 20963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:48.128000 20963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:48.128013 20963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:48.128026 20963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:48.129624 20963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:37209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:37209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:48.129873 20963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:48.130119 20963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:48.135532 20968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:48.137113 20963 server_base.cc:1034] running on GCE node
W20260924 21:16:48.137270 20971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:48.137279 20969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:48.137594 20963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:48.137850 20963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:48.139045 20963 hybrid_clock.cc:648] HybridClock initialized: now 1790284608139030 us; error 26 us; skew 500 ppm
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:48.141515 20963 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:16:48.142072 20963 webserver.cc:460] Webserver started at http://127.20.115.254:43381/ using document root <none> and password file <none>
I20260924 21:16:48.142366 20963 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:48.142443 20963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:48.142601 20963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:48.143661 20963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "da36d83160414931b382f119b884fb98"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "c579c0a4f42099c8cfc44d0bf1eb7e7b"
server_key_iv: "c9685652af7a18db0539712ec82a0b2e"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.144081 20963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "da36d83160414931b382f119b884fb98"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "c579c0a4f42099c8cfc44d0bf1eb7e7b"
server_key_iv: "c9685652af7a18db0539712ec82a0b2e"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.145598 20963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:48.146420 20978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.146608 20963 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:48.146699 20963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "da36d83160414931b382f119b884fb98"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "c579c0a4f42099c8cfc44d0bf1eb7e7b"
server_key_iv: "c9685652af7a18db0539712ec82a0b2e"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.146780 20963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:48.163617 20963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:48.165666 20963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:48.165839 20963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:48.169979 20963 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:37209
I20260924 21:16:48.170017 21030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:37209 every 8 connection(s)
I20260924 21:16:48.170380 20963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:48.171248 21031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:48.173496 21031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98: Bootstrap starting.
I20260924 21:16:48.174028 21031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:48.174258 21031 log.cc:824] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:48.174893 21031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98: No bootstrap required, opened a new log
I20260924 21:16:48.176409 21031 raft_consensus.cc:348] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da36d83160414931b382f119b884fb98" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37209 } }
I20260924 21:16:48.176599 21031 raft_consensus.cc:374] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:48.176625 21031 raft_consensus.cc:729] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da36d83160414931b382f119b884fb98, State: Initialized, Role: FOLLOWER
I20260924 21:16:48.176728 21031 consensus_queue.cc:260] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da36d83160414931b382f119b884fb98" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37209 } }
I20260924 21:16:48.176800 21031 raft_consensus.cc:388] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:48.176872 21031 raft_consensus.cc:482] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:48.176918 21031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:48.177397 21031 raft_consensus.cc:504] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da36d83160414931b382f119b884fb98" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37209 } }
I20260924 21:16:48.177479 21031 leader_election.cc:302] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da36d83160414931b382f119b884fb98; no voters:
I20260924 21:16:48.177598 21031 leader_election.cc:288] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:48.177670 21034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:48.177829 21034 raft_consensus.cc:686] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [term 1 LEADER]: Becoming Leader. State: Replica: da36d83160414931b382f119b884fb98, State: Running, Role: LEADER
I20260924 21:16:48.177870 21031 sys_catalog.cc:564] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:48.177917 21034 consensus_queue.cc:237] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da36d83160414931b382f119b884fb98" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37209 } }
I20260924 21:16:48.178325 21036 sys_catalog.cc:455] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da36d83160414931b382f119b884fb98. Latest consensus state: current_term: 1 leader_uuid: "da36d83160414931b382f119b884fb98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da36d83160414931b382f119b884fb98" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37209 } } }
I20260924 21:16:48.178411 21036 sys_catalog.cc:458] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:48.178526 21035 sys_catalog.cc:455] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da36d83160414931b382f119b884fb98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da36d83160414931b382f119b884fb98" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37209 } } }
I20260924 21:16:48.178638 21035 sys_catalog.cc:458] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:48.178637 21043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:48.179219 21043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:48.179689 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 20963
I20260924 21:16:48.179801 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:16:48.180596 20943 external_mini_cluster.cc:1428] Setting key ef53ea8ede0ab3e2e5ee6721dbc15451
I20260924 21:16:48.180946 21043 catalog_manager.cc:1347] Generated new cluster ID: 17eae44aa2ec4ed0ad287057f5b9a285
I20260924 21:16:48.180990 21043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:48.190179 21043 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:48.190864 21043 catalog_manager.cc:1504] Loading token signing keys...
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:48.195292 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:48.181456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.20.115.254:37209)
0924 21:16:48.181722 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:48.181731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:48.182021 (+ 290us) server_negotiation.cc:407] Connection header received
0924 21:16:48.183666 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:48.183673 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:48.183724 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:48.183856 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:48.185372 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.186387 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.187413 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.187829 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.187962 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:48.191603 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:48.191607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:48.191607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:48.191618 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:48.193374 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.193848 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.193850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.193851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.193939 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.194216 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.194217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.194218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.194435 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:48.194583 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:48.194872 (+ 289us) server_negotiation.cc:299] Negotiation successful
0924 21:16:48.195024 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":75,"threads_started":1}
I20260924 21:16:48.204298 21043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da36d83160414931b382f119b884fb98: Generated new TSK 0
I20260924 21:16:48.204473 21043 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:16:48.240468 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:37209
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:48.339011 21057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:48.339207 21057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:48.339231 21057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:48.340384 21057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:48.340432 21057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:48.340459 21057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:16:48.342020 21057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:37209
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:48.342272 21057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:48.342535 21057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:48.347779 21062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:48.347980 21057 server_base.cc:1034] running on GCE node
W20260924 21:16:48.347783 21063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:48.347851 21065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:48.348439 21057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:48.348719 21057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:48.349861 21057 hybrid_clock.cc:648] HybridClock initialized: now 1790284608349839 us; error 39 us; skew 500 ppm
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:48.352659 21057 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:16:48.353288 21057 webserver.cc:460] Webserver started at http://127.20.115.193:34829/ using document root <none> and password file <none>
I20260924 21:16:48.353542 21057 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:48.353621 21057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:48.353775 21057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:48.354838 21057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "edff9d032e654fbf8788ef9189b59c3d"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "3b8a2f8444e9851581954ae152a27296"
server_key_iv: "69217e5d3524d8730344b6ab92ee996d"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.355201 21057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "edff9d032e654fbf8788ef9189b59c3d"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "3b8a2f8444e9851581954ae152a27296"
server_key_iv: "69217e5d3524d8730344b6ab92ee996d"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.356576 21057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:48.357321 21072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.357483 21057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.357584 21057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "edff9d032e654fbf8788ef9189b59c3d"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "3b8a2f8444e9851581954ae152a27296"
server_key_iv: "69217e5d3524d8730344b6ab92ee996d"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.357663 21057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:48.379019 21057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:48.381316 21057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:48.381479 21057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:48.381708 21057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:48.382007 21057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:48.382061 21057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.382100 21057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:48.382128 21057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.389067 21057 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:42195
I20260924 21:16:48.389446 21057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:16:48.389626 21185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:42195 every 8 connection(s)
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:48.395804 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 21057
I20260924 21:16:48.395910 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:48.396085 20943 external_mini_cluster.cc:1428] Setting key 11a005ae6ec3af3fabbf60cb788858bc
I20260924 21:16:48.398919 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:48.390378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:56343 (local address 127.20.115.254:37209)
0924 21:16:48.390458 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:48.390461 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:48.390683 (+ 222us) server_negotiation.cc:407] Connection header received
0924 21:16:48.391318 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:48.391319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:48.391329 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:48.391371 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:48.392248 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.392748 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.393757 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.393954 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.394011 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:48.396016 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:48.396018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:48.396018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:48.396026 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:48.397516 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.397935 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.397937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.397937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.397994 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.398251 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.398253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.398253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.398387 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:48.398471 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:48.398739 (+ 268us) server_negotiation.cc:299] Negotiation successful
0924 21:16:48.398828 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 21:16:48.399101 21188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:48.390362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:37209 (local address 127.20.115.193:56343)
0924 21:16:48.390542 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:48.390588 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:48.391198 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:48.391427 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:48.391431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:48.391985 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:48.392169 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:48.392174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.392876 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.392877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:48.393677 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:48.393679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.394026 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.394026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:48.394125 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:48.394530 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:48.394537 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:48.395887 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:48.397620 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:48.397627 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:48.397629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:48.397843 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:48.398071 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:48.398072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:48.398072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:48.398182 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:48.398472 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:48.398479 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:48.398610 (+ 131us) client_negotiation.cc:769] Sending connection context
0924 21:16:48.398716 (+ 106us) client_negotiation.cc:240] Negotiation successful
0924 21:16:48.398861 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260924 21:16:48.399612 21186 heartbeater.cc:347] Connected to a master server at 127.20.115.254:37209
I20260924 21:16:48.399714 21186 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:48.399879 21186 heartbeater.cc:511] Master 127.20.115.254:37209 requested a full tablet report, sending...
I20260924 21:16:48.400424 20995 ts_manager.cc:194] Registered new tserver with Master: edff9d032e654fbf8788ef9189b59c3d (127.20.115.193:42195)
I20260924 21:16:48.401433 20995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:56343
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:16:48.422020 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:37209
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:48.503945 21191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:48.504171 21191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:48.504210 21191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:48.505429 21191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:48.505493 21191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:48.505539 21191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:16:48.507002 21191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:37209
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:48.507269 21191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:48.507495 21191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:48.512893 21197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:48.512945 21196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:48.512946 21199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:48.512976 21191 server_base.cc:1034] running on GCE node
I20260924 21:16:48.513279 21191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:48.513517 21191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:48.514686 21191 hybrid_clock.cc:648] HybridClock initialized: now 1790284608514671 us; error 33 us; skew 500 ppm
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:48.517618 21191 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:16:48.518087 21191 webserver.cc:460] Webserver started at http://127.20.115.194:39169/ using document root <none> and password file <none>
I20260924 21:16:48.518389 21191 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:48.518464 21191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:48.518607 21191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:48.519634 21191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "820b26d30ba14923aed0438f296f0eab"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "3baaf68d2d7d133cdacf85388403aa94"
server_key_iv: "47818d14d2277e40d065fbe1f25d565f"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.519984 21191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "820b26d30ba14923aed0438f296f0eab"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "3baaf68d2d7d133cdacf85388403aa94"
server_key_iv: "47818d14d2277e40d065fbe1f25d565f"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.521368 21191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:48.522091 21206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.522256 21191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:48.522356 21191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "820b26d30ba14923aed0438f296f0eab"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "3baaf68d2d7d133cdacf85388403aa94"
server_key_iv: "47818d14d2277e40d065fbe1f25d565f"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.522437 21191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:48.531772 21191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:48.533965 21191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:48.534119 21191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:48.534328 21191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:48.534610 21191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:48.534662 21191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.534700 21191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:48.534730 21191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.540464 21191 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:39535
I20260924 21:16:48.540512 21319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:39535 every 8 connection(s)
I20260924 21:16:48.540858 21191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:16:48.546547 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 21191
I20260924 21:16:48.546622 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:48.546779 20943 external_mini_cluster.cc:1428] Setting key 1180dca707573916f0e5af12ae2980be
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:48.550414 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:48.541565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:57795 (local address 127.20.115.254:37209)
0924 21:16:48.541646 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:48.541648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:48.541821 (+ 173us) server_negotiation.cc:407] Connection header received
0924 21:16:48.542357 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:48.542358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:48.542370 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:48.542425 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:48.543383 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.543891 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.545019 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.545215 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.545297 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:48.547493 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:48.547494 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:48.547494 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:48.547502 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:48.548980 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.549392 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.549393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.549394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.549456 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.549779 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.549780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.549780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.549919 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:48.549981 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:48.550243 (+ 262us) server_negotiation.cc:299] Negotiation successful
0924 21:16:48.550324 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:16:48.550606 21322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:48.541522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:37209 (local address 127.20.115.194:57795)
0924 21:16:48.541695 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:48.541741 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:48.542251 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:48.542501 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:48.542505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:48.543107 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:48.543284 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:48.543289 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.543990 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.543991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:48.544935 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:48.544937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.545307 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.545308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:48.545444 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:48.545852 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:48.545859 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:48.547389 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:48.549078 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:48.549084 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:48.549086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:48.549300 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:48.549530 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:48.549530 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:48.549531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:48.549699 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:48.549973 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:48.549979 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:48.550114 (+ 135us) client_negotiation.cc:769] Sending connection context
0924 21:16:48.550222 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 21:16:48.550373 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260924 21:16:48.551083 21320 heartbeater.cc:347] Connected to a master server at 127.20.115.254:37209
I20260924 21:16:48.551185 21320 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:48.551362 21320 heartbeater.cc:511] Master 127.20.115.254:37209 requested a full tablet report, sending...
I20260924 21:16:48.551733 20995 ts_manager.cc:194] Registered new tserver with Master: 820b26d30ba14923aed0438f296f0eab (127.20.115.194:39535)
I20260924 21:16:48.552106 20995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:57795
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:16:48.571517 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:37209
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:48.650748 21325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:48.650954 21325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:48.650977 21325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:48.652148 21325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:48.652189 21325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:48.652215 21325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:16:48.653656 21325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:37209
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:48.653820 21325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:48.653978 21325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:48.658761 21330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:48.658773 21331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:48.658773 21333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:48.658855 21325 server_base.cc:1034] running on GCE node
I20260924 21:16:48.659282 21325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:48.659518 21325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:48.660688 21325 hybrid_clock.cc:648] HybridClock initialized: now 1790284608660666 us; error 38 us; skew 500 ppm
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:48.663291 21325 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:16:48.663753 21325 webserver.cc:460] Webserver started at http://127.20.115.195:40485/ using document root <none> and password file <none>
I20260924 21:16:48.663978 21325 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:48.664052 21325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:48.664201 21325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:48.665390 21325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "6c5f910f9f8345cba54e0d6af0e6fb3c"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "c9b5fdfdc35bc69a1e383a7e6fe6a09a"
server_key_iv: "b8e1fd7d9e132356b71fe5889f8a97a0"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.665755 21325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "6c5f910f9f8345cba54e0d6af0e6fb3c"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "c9b5fdfdc35bc69a1e383a7e6fe6a09a"
server_key_iv: "b8e1fd7d9e132356b71fe5889f8a97a0"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.667043 21325 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:48.667706 21340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.667879 21325 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:48.667972 21325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "6c5f910f9f8345cba54e0d6af0e6fb3c"
format_stamp: "Formatted at 2026-09-24 21:16:48 on dist-test-slave-cxpd"
server_key: "c9b5fdfdc35bc69a1e383a7e6fe6a09a"
server_key_iv: "b8e1fd7d9e132356b71fe5889f8a97a0"
server_key_version: "encryptionkey@0"
I20260924 21:16:48.668071 21325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:48.680351 21325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:48.682898 21325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:48.683070 21325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:48.683300 21325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:48.683640 21325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:48.683684 21325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.683724 21325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:48.683744 21325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:48.689623 21325 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:34177
I20260924 21:16:48.689669 21453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:34177 every 8 connection(s)
I20260924 21:16:48.689949 21325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:16:48.695820 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 21325
I20260924 21:16:48.695888 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:48.696043 20943 external_mini_cluster.cc:1428] Setting key e39fd7d7e971ecb03412105445cc8ab0
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:48.699312 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:48.690639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:59003 (local address 127.20.115.254:37209)
0924 21:16:48.690702 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:48.690704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:48.690899 (+ 195us) server_negotiation.cc:407] Connection header received
0924 21:16:48.691414 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:48.691416 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:48.691429 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:48.691478 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:48.692390 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.692954 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.694070 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.694273 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.694356 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:48.696529 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:48.696531 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:48.696531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:48.696539 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:48.697964 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.698371 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.698372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.698373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.698433 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.698684 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.698685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.698686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.698801 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:48.698878 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:48.699149 (+ 271us) server_negotiation.cc:299] Negotiation successful
0924 21:16:48.699224 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:16:48.699481 21456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:48.690636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:37209 (local address 127.20.115.195:59003)
0924 21:16:48.690787 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:48.690820 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:48.691302 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:48.691550 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:48.691554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:48.692106 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:48.692290 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:48.692294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.693049 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.693050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:48.693965 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:48.693968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.694364 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.694365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:48.694482 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:48.694969 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:48.694977 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:48.696427 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:48.698065 (+ 1638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:48.698071 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:48.698073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:48.698290 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:48.698502 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:48.698503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:48.698503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:48.698610 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:48.698871 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:48.698876 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:48.699010 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 21:16:48.699112 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 21:16:48.699263 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260924 21:16:48.699957 21454 heartbeater.cc:347] Connected to a master server at 127.20.115.254:37209
I20260924 21:16:48.700032 21454 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:48.700160 21454 heartbeater.cc:511] Master 127.20.115.254:37209 requested a full tablet report, sending...
I20260924 21:16:48.700482 20995 ts_manager.cc:194] Registered new tserver with Master: 6c5f910f9f8345cba54e0d6af0e6fb3c (127.20.115.195:34177)
I20260924 21:16:48.700928 20995 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:59003
I20260924 21:16:48.708005 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
I20260924 21:16:48.730595 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:48.722855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.20.115.193:42195)
0924 21:16:48.723044 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:48.723049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:48.723060 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:48.723084 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:48.723086 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:48.723099 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:48.723167 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:48.723691 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.724394 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.725253 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.725562 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.725619 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:48.727946 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:48.727953 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:48.727953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:48.727959 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:48.729345 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.729722 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.729726 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.729726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.729784 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.729990 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.729991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.729992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.730153 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:48.730331 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:48.730401 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 21:16:48.730481 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
W20260924 21:16:48.730945 21100 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48870
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[20950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284608, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
I20260924 21:16:48.752226 21461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:48.745479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48872 (local address 127.20.115.193:42195)
0924 21:16:48.745539 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:48.745541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:48.745648 (+ 107us) server_negotiation.cc:407] Connection header received
0924 21:16:48.745692 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:48.745693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:48.745703 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:48.745752 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:48.746339 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.746831 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.747518 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:48.747703 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:48.747752 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:48.749987 (+ 2235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:48.749989 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:48.749989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:48.749995 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:48.751181 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.751510 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.751511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.751511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.751558 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:48.751718 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:48.751718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:48.751718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:48.751817 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:48.751944 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:48.752040 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 21:16:48.752120 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:16:48.752753 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 21057
I20260924 21:16:48.757560 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 21191
I20260924 21:16:48.761749 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 21325
I20260924 21:16:48.766042 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 20963
2026-09-24T21:16:48Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1902 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[21468](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[21468](info): set up 2 sockets
Sep 24 21:16:48 dist-test-slave-cxpd krb5kdc[21468](info): commencing operation
krb5kdc: starting...
W20260924 21:16:49.804247 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:16:49 dist-test-slave-cxpd krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:16:49.923588 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:42417
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:36957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:42417
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:50.001221 21481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:50.001451 21481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:50.001488 21481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:50.002621 21481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:50.002692 21481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:50.002717 21481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:50.002737 21481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:50.002768 21481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:50.004204 21481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:42417
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:42417
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:50.004474 21481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:50.004694 21481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:50.009263 21489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:50.009238 21487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:50.009413 21481 server_base.cc:1034] running on GCE node
W20260924 21:16:50.009356 21486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:50.009697 21481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:50.009934 21481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:50.011077 21481 hybrid_clock.cc:648] HybridClock initialized: now 1790284610011058 us; error 40 us; skew 500 ppm
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:50.013458 21481 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:16:50.013936 21481 webserver.cc:460] Webserver started at http://127.20.115.254:43389/ using document root <none> and password file <none>
I20260924 21:16:50.014151 21481 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:50.014217 21481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:50.014366 21481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:50.015379 21481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "085380a75690485ba37d9874e02d9ebe"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "b88584da953c2472c66c7f38006c02a6"
server_key_iv: "50f7e234c5e051b82e2f35b7c28332a3"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.015738 21481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "085380a75690485ba37d9874e02d9ebe"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "b88584da953c2472c66c7f38006c02a6"
server_key_iv: "50f7e234c5e051b82e2f35b7c28332a3"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.017179 21481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:50.017984 21496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.018174 21481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:50.018291 21481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "085380a75690485ba37d9874e02d9ebe"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "b88584da953c2472c66c7f38006c02a6"
server_key_iv: "50f7e234c5e051b82e2f35b7c28332a3"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.018368 21481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:50.043111 21481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:50.045223 21481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:50.045390 21481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:50.049325 21481 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:42417
I20260924 21:16:50.049364 21548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:42417 every 8 connection(s)
I20260924 21:16:50.049726 21481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:50.050544 21549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:50.052747 21549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe: Bootstrap starting.
I20260924 21:16:50.053323 21549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:50.053560 21549 log.cc:824] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:50.054185 21549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe: No bootstrap required, opened a new log
I20260924 21:16:50.055815 21549 raft_consensus.cc:348] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085380a75690485ba37d9874e02d9ebe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42417 } }
I20260924 21:16:50.055949 21549 raft_consensus.cc:374] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:50.055972 21549 raft_consensus.cc:729] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 085380a75690485ba37d9874e02d9ebe, State: Initialized, Role: FOLLOWER
I20260924 21:16:50.056075 21549 consensus_queue.cc:260] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085380a75690485ba37d9874e02d9ebe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42417 } }
I20260924 21:16:50.056159 21549 raft_consensus.cc:388] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:50.056192 21549 raft_consensus.cc:482] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:50.056265 21549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:50.056814 21549 raft_consensus.cc:504] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085380a75690485ba37d9874e02d9ebe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42417 } }
I20260924 21:16:50.056934 21549 leader_election.cc:302] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 085380a75690485ba37d9874e02d9ebe; no voters:
I20260924 21:16:50.057106 21549 leader_election.cc:288] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:50.057160 21552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:50.057331 21552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [term 1 LEADER]: Becoming Leader. State: Replica: 085380a75690485ba37d9874e02d9ebe, State: Running, Role: LEADER
I20260924 21:16:50.057351 21549 sys_catalog.cc:564] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:50.057446 21552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085380a75690485ba37d9874e02d9ebe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42417 } }
I20260924 21:16:50.057871 21553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "085380a75690485ba37d9874e02d9ebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085380a75690485ba37d9874e02d9ebe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42417 } } }
I20260924 21:16:50.057957 21553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:50.057996 21554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 085380a75690485ba37d9874e02d9ebe. Latest consensus state: current_term: 1 leader_uuid: "085380a75690485ba37d9874e02d9ebe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085380a75690485ba37d9874e02d9ebe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42417 } } }
I20260924 21:16:50.058029 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 21481
I20260924 21:16:50.058055 21554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:50.058107 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:16:50.058274 20943 external_mini_cluster.cc:1428] Setting key 92afaef0bf160e58ec4655122a46288c
I20260924 21:16:50.058166 21557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:50.058979 21557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:50.060472 21557 catalog_manager.cc:1347] Generated new cluster ID: 19f374b5907a4b9597b4461e56292677
I20260924 21:16:50.060518 21557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:50.068269 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.058752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35350 (local address 127.20.115.254:42417)
0924 21:16:50.058917 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.058926 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.059607 (+ 681us) server_negotiation.cc:407] Connection header received
0924 21:16:50.059769 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.059775 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.059818 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.059950 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:50.060548 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.061508 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.062248 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.062659 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.062784 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.065091 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:50.065095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:50.065095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:50.065113 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:50.066779 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.067154 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.067157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.067157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.067261 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.067472 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.067473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.067474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.067705 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:50.067868 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.067964 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.068103 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260924 21:16:50.076754 21557 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:50.077302 21557 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:50.086267 21557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 085380a75690485ba37d9874e02d9ebe: Generated new TSK 0
I20260924 21:16:50.086452 21557 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:16:50.124428 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:42417
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:36957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:50.227715 21575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:50.228116 21575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:50.228154 21575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:50.229374 21575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:50.229417 21575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:50.229445 21575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:16:50.230968 21575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:42417
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:50.231222 21575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:50.231485 21575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:50.236475 21583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:50.236490 21581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:50.236503 21580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:50.236649 21575 server_base.cc:1034] running on GCE node
I20260924 21:16:50.236984 21575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:50.237226 21575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:50.238405 21575 hybrid_clock.cc:648] HybridClock initialized: now 1790284610238397 us; error 41 us; skew 500 ppm
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:50.240969 21575 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:16:50.241431 21575 webserver.cc:460] Webserver started at http://127.20.115.193:39491/ using document root <none> and password file <none>
I20260924 21:16:50.241632 21575 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:50.241672 21575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:50.241766 21575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:50.242762 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "33472a5864b04ff18dea909287c0cee9"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "589c4aa53b4aa34a25043756851d1844"
server_key_iv: "e32d870383fad6b980233570f0a5cdc9"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.243076 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "33472a5864b04ff18dea909287c0cee9"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "589c4aa53b4aa34a25043756851d1844"
server_key_iv: "e32d870383fad6b980233570f0a5cdc9"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.244342 21575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:50.245141 21590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.245321 21575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:50.245401 21575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "33472a5864b04ff18dea909287c0cee9"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "589c4aa53b4aa34a25043756851d1844"
server_key_iv: "e32d870383fad6b980233570f0a5cdc9"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.245455 21575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:50.267680 21575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:50.270195 21575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:50.270315 21575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:50.270503 21575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:50.270828 21575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:50.270864 21575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.270925 21575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:50.270953 21575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.276713 21575 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:34047
I20260924 21:16:50.276839 21703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:34047 every 8 connection(s)
I20260924 21:16:50.277158 21575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:16:50.280373 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 21575
I20260924 21:16:50.280454 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:50.280645 20943 external_mini_cluster.cc:1428] Setting key 72b6608f116089600f2e1d7caf37326e
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:50.287835 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.277983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:36249 (local address 127.20.115.254:42417)
0924 21:16:50.278055 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.278057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.278286 (+ 229us) server_negotiation.cc:407] Connection header received
0924 21:16:50.279237 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.279238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.279251 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.279319 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:50.279966 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.280542 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.281829 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.282035 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.282121 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.284618 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:50.284620 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:50.284620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:50.284628 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:50.286276 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.286763 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.286765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.286765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.286825 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.287136 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.287137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.287138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.287268 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:50.287348 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.287639 (+ 291us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.287734 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 21:16:50.288041 21706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.277970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:42417 (local address 127.20.115.193:36249)
0924 21:16:50.278149 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:50.278192 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:50.279106 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:50.279400 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:50.279403 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:50.279665 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:50.279859 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.279863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.280623 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.280624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.281718 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.281722 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.282145 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.282146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.282302 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.283004 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:50.283013 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:50.284513 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:50.286399 (+ 1886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:50.286408 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:50.286410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:50.286676 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:50.286904 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:50.286905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:50.286906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:50.287060 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:50.287339 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:50.287345 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:50.287500 (+ 155us) client_negotiation.cc:769] Sending connection context
0924 21:16:50.287617 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 21:16:50.287782 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260924 21:16:50.288950 21704 heartbeater.cc:347] Connected to a master server at 127.20.115.254:42417
I20260924 21:16:50.289114 21704 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:50.289332 21704 heartbeater.cc:511] Master 127.20.115.254:42417 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
I20260924 21:16:50.290446 21513 ts_manager.cc:194] Registered new tserver with Master: 33472a5864b04ff18dea909287c0cee9 (127.20.115.193:34047)
I20260924 21:16:50.291462 21513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:36249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:16:50.307909 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:42417
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:36957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:50.389360 21709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:50.389567 21709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:50.389591 21709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:50.390770 21709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:50.390817 21709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:50.390844 21709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:16:50.392403 21709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:42417
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:50.392635 21709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:50.392941 21709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:50.398435 21714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:50.398535 21717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:50.398666 21709 server_base.cc:1034] running on GCE node
W20260924 21:16:50.399113 21715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:50.399358 21709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:50.399613 21709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:50.400849 21709 hybrid_clock.cc:648] HybridClock initialized: now 1790284610400830 us; error 30 us; skew 500 ppm
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:50.403319 21709 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:16:50.403797 21709 webserver.cc:460] Webserver started at http://127.20.115.194:32989/ using document root <none> and password file <none>
I20260924 21:16:50.404031 21709 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:50.404076 21709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:50.404219 21709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:50.405333 21709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "5959d32848cf47db8df53d8080190425"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "f4c309823c4f50b2886eede0260067f8"
server_key_iv: "9bf1503f85d102372f37c1d17b85d8d4"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.405692 21709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "5959d32848cf47db8df53d8080190425"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "f4c309823c4f50b2886eede0260067f8"
server_key_iv: "9bf1503f85d102372f37c1d17b85d8d4"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.407086 21709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:50.407866 21724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.408067 21709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:50.408154 21709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "5959d32848cf47db8df53d8080190425"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "f4c309823c4f50b2886eede0260067f8"
server_key_iv: "9bf1503f85d102372f37c1d17b85d8d4"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.408208 21709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:50.428222 21709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:50.430675 21709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:50.430791 21709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:50.430984 21709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:50.431309 21709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:50.431344 21709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.431366 21709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:50.431416 21709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.437182 21709 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:37771
I20260924 21:16:50.437264 21837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:37771 every 8 connection(s)
I20260924 21:16:50.437579 21709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:16:50.442890 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 21709
I20260924 21:16:50.443002 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:50.443181 20943 external_mini_cluster.cc:1428] Setting key dee923a816657a98a244c7ca0c2a4dd2
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:50.447960 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.438415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:59829 (local address 127.20.115.254:42417)
0924 21:16:50.438463 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.438465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.438697 (+ 232us) server_negotiation.cc:407] Connection header received
0924 21:16:50.439675 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.439677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.439690 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.439743 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:50.440376 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.440952 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.441991 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.442187 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.442263 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.444982 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:50.444983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:50.444984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:50.444992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:50.446438 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.446935 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.446936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.446936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.446988 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.447298 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.447298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.447299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.447399 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:50.447450 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.447805 (+ 355us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.447876 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:16:50.448304 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.438387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:42417 (local address 127.20.115.194:59829)
0924 21:16:50.438581 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:50.438614 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:50.439550 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:50.439822 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:50.439825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:50.440076 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:50.440262 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.440267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.441050 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.441051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.441855 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.441857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.442281 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.442282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.442402 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.442956 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:50.442965 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:50.444891 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:50.446526 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:50.446533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:50.446536 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:50.446855 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:50.447081 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:50.447082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:50.447083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:50.447231 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:50.447471 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:50.447478 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:50.447655 (+ 177us) client_negotiation.cc:769] Sending connection context
0924 21:16:50.447785 (+ 130us) client_negotiation.cc:240] Negotiation successful
0924 21:16:50.447977 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":108,"threads_started":1}
I20260924 21:16:50.448844 21838 heartbeater.cc:347] Connected to a master server at 127.20.115.254:42417
I20260924 21:16:50.448948 21838 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:50.449193 21838 heartbeater.cc:511] Master 127.20.115.254:42417 requested a full tablet report, sending...
I20260924 21:16:50.449615 21513 ts_manager.cc:194] Registered new tserver with Master: 5959d32848cf47db8df53d8080190425 (127.20.115.194:37771)
I20260924 21:16:50.449990 21513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:59829
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:16:50.470026 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:42417
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:36957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:50.550278 21843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:50.550475 21843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:50.550500 21843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:50.551573 21843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:50.551606 21843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:50.551632 21843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:16:50.553052 21843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:42417
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:50.553222 21843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:50.553385 21843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:50.558630 21851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:50.558728 21848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:50.560233 21843 server_base.cc:1034] running on GCE node
W20260924 21:16:50.560248 21849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:50.560535 21843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:50.560732 21843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:50.561870 21843 hybrid_clock.cc:648] HybridClock initialized: now 1790284610561861 us; error 31 us; skew 500 ppm
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:50.564333 21843 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:16:50.564851 21843 webserver.cc:460] Webserver started at http://127.20.115.195:36865/ using document root <none> and password file <none>
I20260924 21:16:50.565073 21843 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:50.565137 21843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:50.565294 21843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:50.566313 21843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "cd9515262f144ada82a84775000ccb82"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "27b51323f7e3bbbc942f446838e237b8"
server_key_iv: "07893f2336bc058248b83f8d0ef9e70b"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.566710 21843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "cd9515262f144ada82a84775000ccb82"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "27b51323f7e3bbbc942f446838e237b8"
server_key_iv: "07893f2336bc058248b83f8d0ef9e70b"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.567983 21843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:16:50.568706 21858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.568894 21843 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:50.568989 21843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "cd9515262f144ada82a84775000ccb82"
format_stamp: "Formatted at 2026-09-24 21:16:50 on dist-test-slave-cxpd"
server_key: "27b51323f7e3bbbc942f446838e237b8"
server_key_iv: "07893f2336bc058248b83f8d0ef9e70b"
server_key_version: "encryptionkey@0"
I20260924 21:16:50.569064 21843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:50.579041 21843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:50.581285 21843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:50.581449 21843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:50.581678 21843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:16:50.581976 21843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:50.582028 21843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.582069 21843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:50.582098 21843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:50.588107 21843 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:41181
I20260924 21:16:50.588150 21971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:41181 every 8 connection(s)
I20260924 21:16:50.588444 21843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:50.594628 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 21843
I20260924 21:16:50.594732 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:50.594901 20943 external_mini_cluster.cc:1428] Setting key 0d9f3909ddc99196be056e4212c81d92
I20260924 21:16:50.597510 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.589188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:36739 (local address 127.20.115.254:42417)
0924 21:16:50.589251 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.589253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.589428 (+ 175us) server_negotiation.cc:407] Connection header received
0924 21:16:50.590307 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.590307 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.590316 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.590383 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:50.590951 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.591439 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.592388 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.592575 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.592628 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.594850 (+ 2222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:50.594852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:50.594852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:50.594860 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:50.596301 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.596694 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.596695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.596695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.596742 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.597003 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.597004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.597004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.597107 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:50.597158 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.597378 (+ 220us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.597436 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 21:16:50.597755 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.589145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:42417 (local address 127.20.115.195:36739)
0924 21:16:50.589299 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:50.589342 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:50.590216 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:50.590438 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:50.590442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:50.590694 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:50.590879 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.590883 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.591525 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.591525 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.592312 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.592314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.592649 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.592649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.592745 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.593321 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:50.593329 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:50.594680 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:50.596385 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:50.596391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:50.596393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:50.596630 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:50.596844 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:50.596845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:50.596847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:50.596950 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:50.597163 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:50.597167 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:50.597277 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 21:16:50.597358 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:16:50.597474 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260924 21:16:50.598315 21972 heartbeater.cc:347] Connected to a master server at 127.20.115.254:42417
I20260924 21:16:50.598418 21972 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:50.598595 21972 heartbeater.cc:511] Master 127.20.115.254:42417 requested a full tablet report, sending...
I20260924 21:16:50.598968 21513 ts_manager.cc:194] Registered new tserver with Master: cd9515262f144ada82a84775000ccb82 (127.20.115.195:41181)
I20260924 21:16:50.599339 21513 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:36739
I20260924 21:16:50.606760 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:16:50.614255 21565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.608552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35362 (local address 127.20.115.254:42417)
0924 21:16:50.608618 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.608619 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.608687 (+ 68us) server_negotiation.cc:407] Connection header received
0924 21:16:50.608726 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.608726 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.608735 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.608816 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:50.609339 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.609809 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.610501 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.610671 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.610718 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.611813 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:50.611814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:50.611814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:50.611820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:50.613242 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.613594 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.613595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.613595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.613650 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.613828 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.613828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.613829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.613975 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:50.614051 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.614137 (+ 86us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.614196 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:16:50.616282 21513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:50.617998 21513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:50.624441 21983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.620354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:41181 (local address 127.0.0.1:39404)
0924 21:16:50.620507 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:50.620545 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:50.620610 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:50.621050 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:50.621051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:50.621053 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:50.621107 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.621109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.622493 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.622494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.623282 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.623285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.624226 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.624227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.624288 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.624291 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:50.624343 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:16:50.624372 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":52,"threads_started":1}
I20260924 21:16:50.624604 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.620637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34954 (local address 127.20.115.194:37771)
0924 21:16:50.620822 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.620827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.620837 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:50.620876 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.620878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.620892 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.620987 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:50.621129 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.622062 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.623497 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.624034 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.624125 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.624181 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.624479 (+ 298us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.624521 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260924 21:16:50.624711 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.620355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:34047 (local address 127.0.0.1:38322)
0924 21:16:50.620614 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:50.620623 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:50.620662 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:50.621080 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:50.621081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:50.621083 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:50.621111 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.621112 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.622452 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.622453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.623540 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.623542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.624425 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.624437 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.624511 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.624515 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:50.624549 (+ 34us) client_negotiation.cc:240] Negotiation successful
0924 21:16:50.624597 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":8,"thread_start_us":96,"threads_started":1}
I20260924 21:16:50.624441 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.620399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:37771 (local address 127.0.0.1:34954)
0924 21:16:50.620747 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:50.620757 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:50.620809 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:50.620997 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:50.621000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:50.621003 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:50.621066 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.621068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.622167 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.622168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.623403 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.623406 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.624122 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.624123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.624200 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.624214 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 21:16:50.624281 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 21:16:50.624344 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":70,"threads_started":1}
I20260924 21:16:50.624912 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.620505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.20.115.195:41181)
0924 21:16:50.620854 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.620860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.620872 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:16:50.620914 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.620916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.620931 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.621182 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:50.621258 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.622367 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.623394 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.624116 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.624221 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.624273 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.624370 (+ 97us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.624423 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":91,"threads_started":1}
I20260924 21:16:50.624949 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.620708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38322 (local address 127.20.115.193:34047)
0924 21:16:50.620911 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.620915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.620924 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:16:50.620970 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.620972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.620986 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.621182 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:50.621258 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.622337 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.623602 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.624306 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.624682 (+ 376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.624749 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.624800 (+ 51us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.624859 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":99,"threads_started":1}
I20260924 21:16:50.625674 21772 tablet_service.cc:1467] Processing CreateTablet for tablet c90fd3f02abe4f34bd7db07b9ea5d607 (DEFAULT_TABLE table=test-table [id=5115343b69bc460f86f9af09ba9d618f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:50.625639 21906 tablet_service.cc:1467] Processing CreateTablet for tablet c90fd3f02abe4f34bd7db07b9ea5d607 (DEFAULT_TABLE table=test-table [id=5115343b69bc460f86f9af09ba9d618f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:50.625741 21638 tablet_service.cc:1467] Processing CreateTablet for tablet c90fd3f02abe4f34bd7db07b9ea5d607 (DEFAULT_TABLE table=test-table [id=5115343b69bc460f86f9af09ba9d618f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:50.625995 21906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c90fd3f02abe4f34bd7db07b9ea5d607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:50.625995 21772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c90fd3f02abe4f34bd7db07b9ea5d607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:50.626104 21638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c90fd3f02abe4f34bd7db07b9ea5d607. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:50.628530 21989 tablet_bootstrap.cc:492] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9: Bootstrap starting.
I20260924 21:16:50.629268 21990 tablet_bootstrap.cc:492] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425: Bootstrap starting.
I20260924 21:16:50.629276 21989 tablet_bootstrap.cc:654] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:50.629794 21990 tablet_bootstrap.cc:654] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:50.629786 21989 log.cc:824] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:50.630131 21991 tablet_bootstrap.cc:492] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82: Bootstrap starting.
I20260924 21:16:50.630358 21990 log.cc:824] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:50.630638 21989 tablet_bootstrap.cc:492] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9: No bootstrap required, opened a new log
I20260924 21:16:50.630716 21989 ts_tablet_manager.cc:1397] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:50.630787 21991 tablet_bootstrap.cc:654] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:50.631098 21990 tablet_bootstrap.cc:492] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425: No bootstrap required, opened a new log
I20260924 21:16:50.631176 21990 ts_tablet_manager.cc:1397] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:50.631318 21991 log.cc:824] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:50.632037 21991 tablet_bootstrap.cc:492] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82: No bootstrap required, opened a new log
I20260924 21:16:50.632150 21991 ts_tablet_manager.cc:1397] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:50.632591 21989 raft_consensus.cc:348] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } }
I20260924 21:16:50.632730 21989 raft_consensus.cc:374] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:50.632822 21989 raft_consensus.cc:729] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33472a5864b04ff18dea909287c0cee9, State: Initialized, Role: FOLLOWER
I20260924 21:16:50.632967 21989 consensus_queue.cc:260] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } }
I20260924 21:16:50.633265 21704 heartbeater.cc:503] Master 127.20.115.254:42417 was elected leader, sending a full tablet report...
I20260924 21:16:50.633287 21989 ts_tablet_manager.cc:1428] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:50.633327 21990 raft_consensus.cc:348] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } }
I20260924 21:16:50.633504 21990 raft_consensus.cc:374] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:50.633549 21990 raft_consensus.cc:729] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5959d32848cf47db8df53d8080190425, State: Initialized, Role: FOLLOWER
I20260924 21:16:50.633668 21990 consensus_queue.cc:260] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } }
I20260924 21:16:50.633919 21990 ts_tablet_manager.cc:1428] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:50.634063 21838 heartbeater.cc:503] Master 127.20.115.254:42417 was elected leader, sending a full tablet report...
I20260924 21:16:50.634728 21991 raft_consensus.cc:348] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } }
I20260924 21:16:50.634887 21991 raft_consensus.cc:374] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:50.634925 21991 raft_consensus.cc:729] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd9515262f144ada82a84775000ccb82, State: Initialized, Role: FOLLOWER
I20260924 21:16:50.635036 21991 consensus_queue.cc:260] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } }
I20260924 21:16:50.635331 21972 heartbeater.cc:503] Master 127.20.115.254:42417 was elected leader, sending a full tablet report...
I20260924 21:16:50.635336 21991 ts_tablet_manager.cc:1428] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:50.660640 21997 raft_consensus.cc:482] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:50.660753 21997 raft_consensus.cc:504] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } }
I20260924 21:16:50.661187 21997 leader_election.cc:288] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33472a5864b04ff18dea909287c0cee9 (127.20.115.193:34047), 5959d32848cf47db8df53d8080190425 (127.20.115.194:37771)
I20260924 21:16:50.665148 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.661496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:59355 (local address 127.20.115.193:34047)
0924 21:16:50.661542 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.661544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.661738 (+ 194us) server_negotiation.cc:407] Connection header received
0924 21:16:50.661761 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.661762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.661769 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.661806 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:50.661964 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.662669 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.664179 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.664739 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.664863 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.664873 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.665043 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.665081 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260924 21:16:50.665156 21987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.661496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:56403 (local address 127.20.115.194:37771)
0924 21:16:50.661541 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.661544 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.661552 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:50.661598 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.661599 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.661608 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.661656 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:50.661875 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.662662 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.664179 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.664739 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.664867 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.664875 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.665058 (+ 183us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.665099 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:16:50.665171 21974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.661419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:37771 (local address 127.20.115.195:56403)
0924 21:16:50.661476 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:50.661488 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:50.661531 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:50.661689 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:50.661691 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:50.661692 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:50.661782 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.661783 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.662751 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.662752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.664074 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.664077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.664867 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.664868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.664983 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.664986 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:50.665038 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:16:50.665066 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":56}
I20260924 21:16:50.665180 21998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.661421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:34047 (local address 127.20.115.195:59355)
0924 21:16:50.661623 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:50.661631 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:50.661698 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:50.661831 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:50.661832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:50.661834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:50.661894 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.661896 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.662751 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.662752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.664074 (+ 1322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:50.664077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.664863 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.664864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:50.664945 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.664950 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:16:50.665014 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 21:16:50.665060 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":98,"thread_start_us":111,"threads_started":1}
I20260924 21:16:50.665498 21792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c90fd3f02abe4f34bd7db07b9ea5d607" candidate_uuid: "cd9515262f144ada82a84775000ccb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5959d32848cf47db8df53d8080190425" is_pre_election: true
I20260924 21:16:50.665678 21792 raft_consensus.cc:2454] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd9515262f144ada82a84775000ccb82 in term 0.
I20260924 21:16:50.665934 21860 leader_election.cc:302] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5959d32848cf47db8df53d8080190425, cd9515262f144ada82a84775000ccb82; no voters:
I20260924 21:16:50.665956 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c90fd3f02abe4f34bd7db07b9ea5d607" candidate_uuid: "cd9515262f144ada82a84775000ccb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33472a5864b04ff18dea909287c0cee9" is_pre_election: true
I20260924 21:16:50.666080 21997 raft_consensus.cc:2781] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:50.666100 21658 raft_consensus.cc:2454] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd9515262f144ada82a84775000ccb82 in term 0.
I20260924 21:16:50.666137 21997 raft_consensus.cc:482] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:50.666167 21997 raft_consensus.cc:3037] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:50.666855 21997 raft_consensus.cc:504] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } }
I20260924 21:16:50.667001 21997 leader_election.cc:288] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [CANDIDATE]: Term 1 election: Requested vote from peers 33472a5864b04ff18dea909287c0cee9 (127.20.115.193:34047), 5959d32848cf47db8df53d8080190425 (127.20.115.194:37771)
I20260924 21:16:50.667145 21792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c90fd3f02abe4f34bd7db07b9ea5d607" candidate_uuid: "cd9515262f144ada82a84775000ccb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5959d32848cf47db8df53d8080190425"
I20260924 21:16:50.667225 21792 raft_consensus.cc:3037] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:50.667259 21658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c90fd3f02abe4f34bd7db07b9ea5d607" candidate_uuid: "cd9515262f144ada82a84775000ccb82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33472a5864b04ff18dea909287c0cee9"
I20260924 21:16:50.667316 21658 raft_consensus.cc:3037] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:50.667861 21792 raft_consensus.cc:2454] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd9515262f144ada82a84775000ccb82 in term 1.
I20260924 21:16:50.668016 21860 leader_election.cc:302] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5959d32848cf47db8df53d8080190425, cd9515262f144ada82a84775000ccb82; no voters:
I20260924 21:16:50.668027 21658 raft_consensus.cc:2454] T c90fd3f02abe4f34bd7db07b9ea5d607 P 33472a5864b04ff18dea909287c0cee9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd9515262f144ada82a84775000ccb82 in term 1.
I20260924 21:16:50.668085 21997 raft_consensus.cc:2781] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:50.668241 21997 raft_consensus.cc:686] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [term 1 LEADER]: Becoming Leader. State: Replica: cd9515262f144ada82a84775000ccb82, State: Running, Role: LEADER
I20260924 21:16:50.668336 21997 consensus_queue.cc:237] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } }
I20260924 21:16:50.669100 21513 catalog_manager.cc:5495] T c90fd3f02abe4f34bd7db07b9ea5d607 P cd9515262f144ada82a84775000ccb82 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd9515262f144ada82a84775000ccb82 (127.20.115.195). New cstate: current_term: 1 leader_uuid: "cd9515262f144ada82a84775000ccb82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd9515262f144ada82a84775000ccb82" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "33472a5864b04ff18dea909287c0cee9" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 34047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5959d32848cf47db8df53d8080190425" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37771 } health_report { overall_health: UNKNOWN } } }
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
W20260924 21:16:50.688200 21839 tablet.cc:2369] T c90fd3f02abe4f34bd7db07b9ea5d607 P 5959d32848cf47db8df53d8080190425: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:50.690646 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.682564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38330 (local address 127.20.115.193:34047)
0924 21:16:50.682608 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.682611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.682618 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:16:50.682657 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.682658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.682668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.682716 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:50.683460 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.684245 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.685336 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.685521 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.685580 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.687712 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:50.687721 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:50.687721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:50.687727 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:50.689343 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.689713 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.689716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.689717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.689790 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.690219 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.690220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.690220 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.690365 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:50.690482 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.690516 (+ 34us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.690580 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
I20260924 21:16:50.713035 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.706241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38340 (local address 127.20.115.193:34047)
0924 21:16:50.706325 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.706327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.706408 (+ 81us) server_negotiation.cc:407] Connection header received
0924 21:16:50.706436 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.706437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.706447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.706506 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:50.707066 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.707569 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.708319 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.708520 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.708575 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.710712 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:50.710713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:50.710714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:50.710720 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:50.712018 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.712326 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.712327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.712327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.712380 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.712555 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.712556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.712556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.712643 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:50.712714 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.712898 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.712962 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[21468](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284610, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
I20260924 21:16:50.734211 21988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:50.727797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.20.115.193:34047)
0924 21:16:50.727871 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:50.727873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:50.727901 (+ 28us) server_negotiation.cc:407] Connection header received
0924 21:16:50.727974 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:50.727975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:50.727984 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:50.728035 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:50.728540 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.729073 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.729764 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:50.729938 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:50.729986 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:50.732107 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:50.732109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:50.732109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:50.732116 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:50.733328 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.733634 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.733635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.733635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.733684 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:50.733843 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:50.733843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:50.733844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:50.733928 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:50.733999 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:50.734093 (+ 94us) server_negotiation.cc:299] Negotiation successful
0924 21:16:50.734147 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260924 21:16:50.734692 21618 tablet_service.cc:2908] Rejecting scan request for tablet c90fd3f02abe4f34bd7db07b9ea5d607: Uninitialized: clean time has not yet been initialized
I20260924 21:16:50.735610 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 21575
I20260924 21:16:50.740947 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 21709
I20260924 21:16:50.745252 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 21843
I20260924 21:16:50.749720 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 21481
2026-09-24T21:16:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (1983 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[22013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[22013](info): set up 2 sockets
Sep 24 21:16:50 dist-test-slave-cxpd krb5kdc[22013](info): commencing operation
krb5kdc: starting...
W20260924 21:16:51.788278 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:16:51 dist-test-slave-cxpd krb5kdc[22013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:16:51.903512 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45461
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:45461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:51.982527 22026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:51.982721 22026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:51.982743 22026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:51.983891 22026 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:51.983933 22026 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 21:16:51.983946 22026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:51.983960 22026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:51.983973 22026 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:51.983985 22026 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:51.985448 22026 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:45461
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:51.985687 22026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:51.985921 22026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:51.990622 22034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:51.990744 22032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:51.990634 22031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:51.990888 22026 server_base.cc:1034] running on GCE node
I20260924 21:16:51.991185 22026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:51.991411 22026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:51.992556 22026 hybrid_clock.cc:648] HybridClock initialized: now 1790284611992532 us; error 43 us; skew 500 ppm
Sep 24 21:16:51 dist-test-slave-cxpd krb5kdc[22013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284611, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:52.000008 22026 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:16:52.000478 22026 webserver.cc:460] Webserver started at http://127.20.115.254:41405/ using document root <none> and password file <none>
I20260924 21:16:52.000694 22026 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:52.000761 22026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:52.000919 22026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:52.001914 22026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "890ea9271891423583054c1d3330aae3"
format_stamp: "Formatted at 2026-09-24 21:16:51 on dist-test-slave-cxpd"
server_key: "dc621911eda0735ef86811aa83f2dfac"
server_key_iv: "b844d397d2c83f5e663559a71e44cfa0"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.002262 22026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "890ea9271891423583054c1d3330aae3"
format_stamp: "Formatted at 2026-09-24 21:16:51 on dist-test-slave-cxpd"
server_key: "dc621911eda0735ef86811aa83f2dfac"
server_key_iv: "b844d397d2c83f5e663559a71e44cfa0"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.003513 22026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:52.004190 22041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.004355 22026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:52.004460 22026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "890ea9271891423583054c1d3330aae3"
format_stamp: "Formatted at 2026-09-24 21:16:51 on dist-test-slave-cxpd"
server_key: "dc621911eda0735ef86811aa83f2dfac"
server_key_iv: "b844d397d2c83f5e663559a71e44cfa0"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.004539 22026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:52.022935 22026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:52.025189 22026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:52.025344 22026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:52.029933 22026 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:45461
I20260924 21:16:52.029996 22103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:45461 every 8 connection(s)
I20260924 21:16:52.030283 22026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:52.030879 22104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:52.033128 22104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3: Bootstrap starting.
I20260924 21:16:52.033701 22104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:52.033995 22104 log.cc:824] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:52.034634 22104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3: No bootstrap required, opened a new log
I20260924 21:16:52.036643 22104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ea9271891423583054c1d3330aae3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45461 } }
I20260924 21:16:52.036825 22104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:52.036866 22104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 890ea9271891423583054c1d3330aae3, State: Initialized, Role: FOLLOWER
I20260924 21:16:52.037020 22104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ea9271891423583054c1d3330aae3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45461 } }
I20260924 21:16:52.037086 22104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:52.037108 22104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:52.037158 22104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:52.037676 22104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ea9271891423583054c1d3330aae3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45461 } }
I20260924 21:16:52.037788 22104 leader_election.cc:302] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 890ea9271891423583054c1d3330aae3; no voters:
I20260924 21:16:52.037961 22104 leader_election.cc:288] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:52.037997 22107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:52.038208 22107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [term 1 LEADER]: Becoming Leader. State: Replica: 890ea9271891423583054c1d3330aae3, State: Running, Role: LEADER
I20260924 21:16:52.038216 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 22026
I20260924 21:16:52.038300 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:16:52.038229 22104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:52.038389 22107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ea9271891423583054c1d3330aae3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45461 } }
I20260924 21:16:52.038463 20943 external_mini_cluster.cc:1428] Setting key f648333bc78a5974d2423b80a9d8f586
I20260924 21:16:52.039623 22109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 890ea9271891423583054c1d3330aae3. Latest consensus state: current_term: 1 leader_uuid: "890ea9271891423583054c1d3330aae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ea9271891423583054c1d3330aae3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45461 } } }
I20260924 21:16:52.039597 22108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "890ea9271891423583054c1d3330aae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "890ea9271891423583054c1d3330aae3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45461 } } }
I20260924 21:16:52.039781 22109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:52.039819 22108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3 [sys.catalog]: This master's current role is: LEADER
W20260924 21:16:52.040208 22124 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:16:52.040272 22124 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:16:52.040369 22125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:52.040565 22125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:52.042104 22125 catalog_manager.cc:1347] Generated new cluster ID: ada99c78ed3f43d494ad4d5a8933fd3d
I20260924 21:16:52.042157 22125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:16:52 dist-test-slave-cxpd krb5kdc[22013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:52.048960 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.039007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.20.115.254:45461)
0924 21:16:52.039256 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.039266 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.039304 (+ 38us) server_negotiation.cc:407] Connection header received
0924 21:16:52.039880 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.039888 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.039949 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.040100 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:52.040986 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.042254 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.042995 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.043401 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.043520 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.045801 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:52.045805 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:52.045805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:52.045816 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:52.047454 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.047820 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.047823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.047824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.047930 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.048137 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.048138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.048138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.048358 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:52.048531 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.048595 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.048732 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":72,"threads_started":1}
I20260924 21:16:52.066838 22125 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:52.067539 22125 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:52.071187 22125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3: Generated new TSK 0
I20260924 21:16:52.071362 22125 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:16:52.103394 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45461
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:52.184288 22130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:52.184485 22130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:52.184509 22130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:52.185685 22130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:52.185727 22130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:52.185747 22130 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:16:52.185793 22130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:16:52.187333 22130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.115.254:45461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:52.187577 22130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:52.187796 22130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:52.192840 22138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:52.192836 22135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:52.192831 22136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:52.193099 22130 server_base.cc:1034] running on GCE node
I20260924 21:16:52.193363 22130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:52.193621 22130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:52.194777 22130 hybrid_clock.cc:648] HybridClock initialized: now 1790284612194765 us; error 29 us; skew 500 ppm
Sep 24 21:16:52 dist-test-slave-cxpd krb5kdc[22013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284612, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:52.197420 22130 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:16:52.197922 22130 webserver.cc:460] Webserver started at http://127.20.115.193:33237/ using document root <none> and password file <none>
I20260924 21:16:52.198164 22130 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:52.198231 22130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:52.198388 22130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:52.199388 22130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "fb299619d2254711bbb9f2d794e2e6be"
format_stamp: "Formatted at 2026-09-24 21:16:52 on dist-test-slave-cxpd"
server_key: "c442343f18c28c5c37529b39d984900a"
server_key_iv: "ba94f0c94166246e0652d1dfdcbf6008"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.199740 22130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "fb299619d2254711bbb9f2d794e2e6be"
format_stamp: "Formatted at 2026-09-24 21:16:52 on dist-test-slave-cxpd"
server_key: "c442343f18c28c5c37529b39d984900a"
server_key_iv: "ba94f0c94166246e0652d1dfdcbf6008"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.201063 22130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:52.201764 22145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.201951 22130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:52.202049 22130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "fb299619d2254711bbb9f2d794e2e6be"
format_stamp: "Formatted at 2026-09-24 21:16:52 on dist-test-slave-cxpd"
server_key: "c442343f18c28c5c37529b39d984900a"
server_key_iv: "ba94f0c94166246e0652d1dfdcbf6008"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.202128 22130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:52.225976 22130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:52.227779 22130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:52.227957 22130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:52.228494 22130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:52.228554 22130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.228611 22130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:52.228641 22130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:16:52 dist-test-slave-cxpd krb5kdc[22013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284612, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:52.236164 22130 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:46627
I20260924 21:16:52.236238 22260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:46627 every 8 connection(s)
I20260924 21:16:52.236559 22130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:16:52.238962 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.228898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:38267 (local address 127.20.115.254:45461)
0924 21:16:52.228968 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.228970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.229219 (+ 249us) server_negotiation.cc:407] Connection header received
0924 21:16:52.229916 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.229917 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.229930 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.229973 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:52.230871 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.231384 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.233118 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.233341 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.233411 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.235857 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:52.235858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:52.235859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:52.235901 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:52.237648 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.238044 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.238044 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.238045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.238096 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.238340 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.238341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.238341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.238449 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:52.238499 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.238728 (+ 229us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.238853 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:16:52.238950 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 22130
I20260924 21:16:52.239054 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:52.239089 22154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.228871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45461 (local address 127.20.115.193:38267)
0924 21:16:52.229072 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.229117 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.229809 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.230053 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.230057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.230460 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:52.230761 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.230768 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.231480 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.231481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.233017 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.233021 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.233438 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.233439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.233605 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.234172 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:52.234181 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:52.235738 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:52.237737 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.237742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.237744 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.237976 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.238138 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.238139 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.238139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.238286 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.238506 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:52.238510 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:52.238633 (+ 123us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.238712 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.238897 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260924 21:16:52.239240 20943 external_mini_cluster.cc:1428] Setting key ee681e1532e8a6761d78b113f3aeba20
I20260924 21:16:52.239660 22261 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45461
I20260924 21:16:52.239735 22261 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:52.239954 22261 heartbeater.cc:511] Master 127.20.115.254:45461 requested a full tablet report, sending...
I20260924 21:16:52.240494 22058 ts_manager.cc:194] Registered new tserver with Master: fb299619d2254711bbb9f2d794e2e6be (127.20.115.193:46627)
I20260924 21:16:52.241411 22058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:38267
I20260924 21:16:52.247906 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.241822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:45187 (local address 127.20.115.254:45461)
0924 21:16:52.241865 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.241867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.242066 (+ 199us) server_negotiation.cc:407] Connection header received
0924 21:16:52.242115 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.242116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.242125 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.242184 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:52.242567 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.243305 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.244208 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.244515 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.244572 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.245599 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:52.245600 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:52.245600 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:52.245605 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:52.246856 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.247239 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.247239 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.247240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.247286 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.247489 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.247489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.247489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.247577 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:52.247625 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.247793 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.247849 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:16:52.248013 22268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.241768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45461 (local address 127.20.115.193:45187)
0924 21:16:52.241950 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.241963 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.242059 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.242190 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.242191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.242420 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:52.242486 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.242488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.243496 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.243497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.244120 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.244122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.244595 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.244596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.244689 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.245075 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:52.245081 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:52.245497 (+ 416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:52.246937 (+ 1440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.246938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.246939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.247157 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.247347 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.247348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.247348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.247420 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.247643 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:52.247644 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:52.247690 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.247771 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.247882 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":126,"threads_started":1}
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:16:52.266000 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45461
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:52.344898 22270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:52.345136 22270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:52.345177 22270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:52.346333 22270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:52.346400 22270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:52.346434 22270 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:16:52.346474 22270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:16:52.347878 22270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.115.254:45461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:52.348155 22270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:52.348376 22270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:52.353221 22275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:52.353230 22276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:52.353229 22278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:52.353335 22270 server_base.cc:1034] running on GCE node
I20260924 21:16:52.353502 22270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:52.353698 22270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:52.354846 22270 hybrid_clock.cc:648] HybridClock initialized: now 1790284612354825 us; error 37 us; skew 500 ppm
Sep 24 21:16:52 dist-test-slave-cxpd krb5kdc[22013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284612, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:52.357203 22270 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:16:52.357641 22270 webserver.cc:460] Webserver started at http://127.20.115.194:42349/ using document root <none> and password file <none>
I20260924 21:16:52.357851 22270 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:52.357893 22270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:52.358095 22270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:52.359093 22270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "c83614a70cdc4346b70f189d08530d87"
format_stamp: "Formatted at 2026-09-24 21:16:52 on dist-test-slave-cxpd"
server_key: "f21e29989730b160f67e8d15b7dd308d"
server_key_iv: "873673ab5deff92c2220b99c17a5bb4a"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.359462 22270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "c83614a70cdc4346b70f189d08530d87"
format_stamp: "Formatted at 2026-09-24 21:16:52 on dist-test-slave-cxpd"
server_key: "f21e29989730b160f67e8d15b7dd308d"
server_key_iv: "873673ab5deff92c2220b99c17a5bb4a"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.360898 22270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:16:52.361611 22285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.361799 22270 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:52.361887 22270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "c83614a70cdc4346b70f189d08530d87"
format_stamp: "Formatted at 2026-09-24 21:16:52 on dist-test-slave-cxpd"
server_key: "f21e29989730b160f67e8d15b7dd308d"
server_key_iv: "873673ab5deff92c2220b99c17a5bb4a"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.361961 22270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:52.377223 22270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:52.379114 22270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:52.379272 22270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:52.379832 22270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:52.379871 22270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.379894 22270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:52.379906 22270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:16:52 dist-test-slave-cxpd krb5kdc[22013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284612, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:52.387351 22270 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:37579
I20260924 21:16:52.387415 22400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:37579 every 8 connection(s)
I20260924 21:16:52.387727 22270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:16:52.389957 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.380348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:49889 (local address 127.20.115.254:45461)
0924 21:16:52.380407 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.380409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.380608 (+ 199us) server_negotiation.cc:407] Connection header received
0924 21:16:52.381211 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.381213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.381227 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.381273 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:52.381928 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.382489 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.383546 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.383841 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.383925 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.386937 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:52.386939 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:52.386939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:52.386947 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:52.388503 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.388978 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.388980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.388980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.389044 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.389323 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.389324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.389324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.389437 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:52.389522 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.389789 (+ 267us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.389869 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:16:52.390136 22294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.380304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45461 (local address 127.20.115.194:49889)
0924 21:16:52.380472 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.380520 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.381094 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.381361 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.381364 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.381597 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:52.381830 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.381835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.382584 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.382585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.383432 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.383434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.383935 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.383936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.384094 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.384589 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:52.384601 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:52.386837 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:52.388597 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.388604 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.388606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.388893 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.389114 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.389115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.389115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.389248 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.389505 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:52.389512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:52.389653 (+ 141us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.389753 (+ 100us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.389905 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260924 21:16:52.390534 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 22270
I20260924 21:16:52.390626 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:52.390699 22401 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45461
I20260924 21:16:52.390787 22401 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:52.390816 20943 external_mini_cluster.cc:1428] Setting key d83403b2bd1a9b4adc54a73f9df71aa7
I20260924 21:16:52.390985 22401 heartbeater.cc:511] Master 127.20.115.254:45461 requested a full tablet report, sending...
I20260924 21:16:52.391462 22058 ts_manager.cc:194] Registered new tserver with Master: c83614a70cdc4346b70f189d08530d87 (127.20.115.194:37579)
I20260924 21:16:52.391852 22058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:49889
I20260924 21:16:52.398494 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.392338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:55765 (local address 127.20.115.254:45461)
0924 21:16:52.392381 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.392383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.393039 (+ 656us) server_negotiation.cc:407] Connection header received
0924 21:16:52.393126 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.393127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.393137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.393187 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:52.393481 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.394173 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.394901 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.395099 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.395183 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.396078 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:52.396079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:52.396080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:52.396086 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:52.397360 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.397705 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.397707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.397707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.397764 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.397964 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.397965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.397965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.398106 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:52.398180 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.398343 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.398409 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:16:52.398511 22407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.392324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45461 (local address 127.20.115.194:55765)
0924 21:16:52.392967 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.392977 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.393051 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.393189 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.393190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.393360 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:52.393407 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.393408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.394275 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.394276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.394780 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.394782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.395184 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.395185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.395262 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.395716 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:52.395720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:52.395980 (+ 260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:52.397449 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.397450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.397450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.397619 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.397830 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.397831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.397832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.397888 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.398182 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:52.398183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:52.398228 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.398306 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.398407 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:16:52.419958 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45461
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:52.500721 22410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:52.501022 22410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:52.501050 22410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:52.502214 22410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:52.502254 22410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:52.502274 22410 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:16:52.502291 22410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:16:52.503648 22410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.115.254:45461
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:52.503825 22410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:52.503988 22410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:52.509123 22416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:52.509126 22410 server_base.cc:1034] running on GCE node
W20260924 21:16:52.509091 22415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:52.509126 22418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:52.509481 22410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:52.509732 22410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:52.510896 22410 hybrid_clock.cc:648] HybridClock initialized: now 1790284612510892 us; error 34 us; skew 500 ppm
Sep 24 21:16:52 dist-test-slave-cxpd krb5kdc[22013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284612, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:52.513337 22410 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:16:52.513801 22410 webserver.cc:460] Webserver started at http://127.20.115.195:34157/ using document root <none> and password file <none>
I20260924 21:16:52.514040 22410 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:52.514109 22410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:52.514259 22410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:52.515286 22410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "7a944235b6bd487e97691275d9e144bb"
format_stamp: "Formatted at 2026-09-24 21:16:52 on dist-test-slave-cxpd"
server_key: "5ddbf4808bfb87eb8d1c18e08da237e2"
server_key_iv: "e8e3337b649e06937a057f1ec640d980"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.515648 22410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "7a944235b6bd487e97691275d9e144bb"
format_stamp: "Formatted at 2026-09-24 21:16:52 on dist-test-slave-cxpd"
server_key: "5ddbf4808bfb87eb8d1c18e08da237e2"
server_key_iv: "e8e3337b649e06937a057f1ec640d980"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.516983 22410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:16:52.517707 22425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.517879 22410 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.517982 22410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "7a944235b6bd487e97691275d9e144bb"
format_stamp: "Formatted at 2026-09-24 21:16:52 on dist-test-slave-cxpd"
server_key: "5ddbf4808bfb87eb8d1c18e08da237e2"
server_key_iv: "e8e3337b649e06937a057f1ec640d980"
server_key_version: "encryptionkey@0"
I20260924 21:16:52.518065 22410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:52.533581 22410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:52.535972 22410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:52.536149 22410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:52.536720 22410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:52.536778 22410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.536871 22410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:52.536899 22410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:16:52 dist-test-slave-cxpd krb5kdc[22013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284612, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:52.544381 22410 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:40211
I20260924 21:16:52.544605 22540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:40211 every 8 connection(s)
I20260924 21:16:52.544967 22410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:16:52.547152 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.537279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:39587 (local address 127.20.115.254:45461)
0924 21:16:52.537334 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.537336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.537545 (+ 209us) server_negotiation.cc:407] Connection header received
0924 21:16:52.538103 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.538104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.538119 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.538170 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:52.538949 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.539798 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.541536 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.541734 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.541817 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.544019 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:52.544021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:52.544021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:52.544030 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:52.545661 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.546118 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.546119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.546120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.546180 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.546446 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.546447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.546447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.546573 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:52.546647 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.546976 (+ 329us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.547060 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 21:16:52.547374 22434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.537266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45461 (local address 127.20.115.195:39587)
0924 21:16:52.537420 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.537456 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.537990 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.538244 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.538247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.538642 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:52.538853 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.538860 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.539879 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.539880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.541425 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.541429 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.541835 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.541836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.541976 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.542497 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:52.542506 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:52.543916 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:52.545762 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.545768 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.545771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.546034 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.546259 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.546260 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.546260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.546368 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.546653 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:52.546659 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:52.546810 (+ 151us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.546940 (+ 130us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.547120 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260924 21:16:52.547868 22541 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45461
I20260924 21:16:52.547957 22541 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:52.548151 22541 heartbeater.cc:511] Master 127.20.115.254:45461 requested a full tablet report, sending...
I20260924 21:16:52.548830 22057 ts_manager.cc:194] Registered new tserver with Master: 7a944235b6bd487e97691275d9e144bb (127.20.115.195:40211)
I20260924 21:16:52.549330 22057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:39587
I20260924 21:16:52.555127 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 22410
I20260924 21:16:52.555243 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:52.555335 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.549501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:53235 (local address 127.20.115.254:45461)
0924 21:16:52.549563 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.549569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.549576 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:16:52.549593 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.549594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.549602 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.549693 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:52.550189 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.550680 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.551412 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.551615 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.551699 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.552660 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:52.552661 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:52.552662 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:52.552669 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:52.554156 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.554529 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.554530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.554531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.554584 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.554790 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.554791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.554791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.554925 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:52.555000 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.555172 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.555242 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:16:52.555351 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.548920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45461 (local address 127.20.115.195:53235)
0924 21:16:52.549057 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.549068 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.549118 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.549702 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.549703 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.550023 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:52.550078 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.550081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.550784 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.550785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.551309 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.551311 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.551701 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.551702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.551777 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.552262 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:52.552267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:52.552553 (+ 286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:52.554263 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.554264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.554265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.554443 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.554659 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:52.554659 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:52.554660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:52.554725 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:52.555003 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:52.555004 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:52.555051 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.555136 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.555245 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260924 21:16:52.555428 20943 external_mini_cluster.cc:1428] Setting key 77f1deaaa1d1adc1a73632caa7881dc8
I20260924 21:16:52.556298 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 21:16:52 dist-test-slave-cxpd krb5kdc[22013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:16:52 dist-test-slave-cxpd krb5kdc[22013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:52.578696 22117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.571696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53296 (local address 127.20.115.254:45461)
0924 21:16:52.571758 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.571760 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.571777 (+ 17us) server_negotiation.cc:407] Connection header received
0924 21:16:52.571802 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.571803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.571812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.571863 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:52.572394 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.572938 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.573671 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.573854 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.573908 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.576121 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:52.576122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:52.576123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:52.576129 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:52.577536 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.577906 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.577907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.577907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.577980 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:52.578230 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:52.578231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:52.578231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:52.578361 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:52.578451 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.578567 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.578630 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:16:52.580569 22058 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:52.582345 22058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:52.588829 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.584949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34144 (local address 127.20.115.194:37579)
0924 21:16:52.585233 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.585239 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.585255 (+ 16us) server_negotiation.cc:407] Connection header received
0924 21:16:52.585309 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.585312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.585329 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.585420 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:52.585623 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.586376 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.587717 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.588380 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.588468 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.588528 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.588673 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.588711 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":81,"threads_started":1}
I20260924 21:16:52.588788 22559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.584680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:37579 (local address 127.0.0.1:34144)
0924 21:16:52.584908 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.584951 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.585021 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.585423 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.585463 (+ 40us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.585466 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:52.585537 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.585539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.586467 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.586468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.587624 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.587626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.588465 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.588466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.588549 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.588570 (+ 21us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.588647 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.588701 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":1,"thread_start_us":50,"threads_started":1}
I20260924 21:16:52.589111 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.584719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:46627 (local address 127.0.0.1:52398)
0924 21:16:52.585088 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.585101 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.585154 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.585696 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.585697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.585699 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:52.585729 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.585730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.586956 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.586957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.587874 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.587876 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.588782 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.588783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.588926 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.588930 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.588963 (+ 33us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.589002 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":2,"thread_start_us":135,"threads_started":1}
I20260924 21:16:52.589219 22558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.584970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:40211 (local address 127.0.0.1:55340)
0924 21:16:52.585109 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.585118 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.585159 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.585580 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.585581 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.585583 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:52.585640 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.585642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.586933 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.586934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.588038 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.588040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.588979 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.588979 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.589027 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.589030 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.589085 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.589116 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":32,"thread_start_us":82,"threads_started":1}
I20260924 21:16:52.589247 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.585190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55340 (local address 127.20.115.195:40211)
0924 21:16:52.585364 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.585368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.585376 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:52.585413 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.585415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.585428 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.585500 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:52.585721 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.586832 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.588116 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.588839 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.588897 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.588947 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.589101 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.589145 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":80,"threads_started":1}
I20260924 21:16:52.589464 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.584875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52398 (local address 127.20.115.193:46627)
0924 21:16:52.585535 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.585539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.585547 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:52.585582 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.585584 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.585597 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.585801 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:52.585858 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.586894 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.587958 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.588680 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.589223 (+ 543us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.589285 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.589311 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.589352 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":86,"threads_started":1}
I20260924 21:16:52.590003 22461 tablet_service.cc:1467] Processing CreateTablet for tablet 649369e5fda54bc78a6723132f47ab16 (DEFAULT_TABLE table=test-table [id=4be45164eeb14271a18f80210e346748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:52.590183 22195 tablet_service.cc:1467] Processing CreateTablet for tablet 649369e5fda54bc78a6723132f47ab16 (DEFAULT_TABLE table=test-table [id=4be45164eeb14271a18f80210e346748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:52.590410 22461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649369e5fda54bc78a6723132f47ab16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:52.590464 22195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649369e5fda54bc78a6723132f47ab16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:52.593995 22564 tablet_bootstrap.cc:492] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: Bootstrap starting.
I20260924 21:16:52.594244 22565 tablet_bootstrap.cc:492] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: Bootstrap starting.
I20260924 21:16:52.594903 22565 tablet_bootstrap.cc:654] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:52.595021 22564 tablet_bootstrap.cc:654] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:52.594949 22335 tablet_service.cc:1467] Processing CreateTablet for tablet 649369e5fda54bc78a6723132f47ab16 (DEFAULT_TABLE table=test-table [id=4be45164eeb14271a18f80210e346748]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:52.595259 22565 log.cc:824] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:52.595310 22335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 649369e5fda54bc78a6723132f47ab16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:52.595361 22564 log.cc:824] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:52.596028 22564 tablet_bootstrap.cc:492] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: No bootstrap required, opened a new log
I20260924 21:16:52.596028 22565 tablet_bootstrap.cc:492] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: No bootstrap required, opened a new log
I20260924 21:16:52.596073 22564 ts_tablet_manager.cc:1397] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:52.596115 22565 ts_tablet_manager.cc:1397] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:52.597707 22564 raft_consensus.cc:348] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.597878 22564 raft_consensus.cc:374] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:52.597944 22564 raft_consensus.cc:729] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a944235b6bd487e97691275d9e144bb, State: Initialized, Role: FOLLOWER
I20260924 21:16:52.598085 22564 consensus_queue.cc:260] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.598145 22565 raft_consensus.cc:348] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.598351 22565 raft_consensus.cc:374] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:52.598373 22564 ts_tablet_manager.cc:1428] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:16:52.598387 22565 raft_consensus.cc:729] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb299619d2254711bbb9f2d794e2e6be, State: Initialized, Role: FOLLOWER
I20260924 21:16:52.598500 22565 consensus_queue.cc:260] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.598770 22565 ts_tablet_manager.cc:1428] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:52.598938 22541 heartbeater.cc:503] Master 127.20.115.254:45461 was elected leader, sending a full tablet report...
I20260924 21:16:52.598984 22570 tablet_bootstrap.cc:492] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: Bootstrap starting.
I20260924 21:16:52.599252 22261 heartbeater.cc:503] Master 127.20.115.254:45461 was elected leader, sending a full tablet report...
I20260924 21:16:52.599669 22570 tablet_bootstrap.cc:654] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:52.600072 22570 log.cc:824] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:52.600941 22570 tablet_bootstrap.cc:492] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: No bootstrap required, opened a new log
I20260924 21:16:52.601022 22570 ts_tablet_manager.cc:1397] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:52.602371 22570 raft_consensus.cc:348] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.602521 22570 raft_consensus.cc:374] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:52.602567 22570 raft_consensus.cc:729] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c83614a70cdc4346b70f189d08530d87, State: Initialized, Role: FOLLOWER
I20260924 21:16:52.602679 22570 consensus_queue.cc:260] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.602936 22570 ts_tablet_manager.cc:1428] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:52.603021 22401 heartbeater.cc:503] Master 127.20.115.254:45461 was elected leader, sending a full tablet report...
W20260924 21:16:52.638394 22402 tablet.cc:2369] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:16:52.737473 22262 tablet.cc:2369] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:16:52.795593 22542 tablet.cc:2369] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:52.860301 22569 raft_consensus.cc:482] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:52.860441 22569 raft_consensus.cc:504] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.860795 22569 leader_election.cc:288] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c83614a70cdc4346b70f189d08530d87 (127.20.115.194:37579), 7a944235b6bd487e97691275d9e144bb (127.20.115.195:40211)
I20260924 21:16:52.864005 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.861151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:33471 (local address 127.20.115.194:37579)
0924 21:16:52.861222 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.861224 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.861283 (+ 59us) server_negotiation.cc:407] Connection header received
0924 21:16:52.861347 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.861348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.861360 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.861445 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:52.861780 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.862276 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.863273 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.863635 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.863721 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.863729 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.863892 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.863931 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 21:16:52.864017 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.861393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:37951 (local address 127.20.115.195:40211)
0924 21:16:52.861513 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.861515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.861524 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:16:52.861553 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.861554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.861563 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.861639 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:52.861853 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.862358 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.863274 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.863637 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.863719 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.863727 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.863900 (+ 173us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.863945 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260924 21:16:52.864060 22573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.861068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:37579 (local address 127.20.115.193:33471)
0924 21:16:52.861199 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.861212 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.861282 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.861615 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.861616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.861619 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:52.861694 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.861696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.862360 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.862361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.863175 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.863177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.863732 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.863733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.863815 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.863819 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.863887 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.863929 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":23,"thread_start_us":72,"threads_started":1}
I20260924 21:16:52.864094 22574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.861260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:40211 (local address 127.20.115.193:37951)
0924 21:16:52.861394 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:52.861424 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:52.861473 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:52.861661 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:52.861661 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:52.861664 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:52.861760 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.861763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.862440 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.862441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.863173 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:52.863175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.863731 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.863732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:52.863817 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.863820 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:52.863883 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 21:16:52.863923 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":2,"thread_start_us":56,"threads_started":1}
I20260924 21:16:52.864439 22355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "649369e5fda54bc78a6723132f47ab16" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83614a70cdc4346b70f189d08530d87" is_pre_election: true
I20260924 21:16:52.864521 22495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "649369e5fda54bc78a6723132f47ab16" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a944235b6bd487e97691275d9e144bb" is_pre_election: true
I20260924 21:16:52.864603 22355 raft_consensus.cc:2454] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 0.
I20260924 21:16:52.864641 22495 raft_consensus.cc:2454] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 0.
I20260924 21:16:52.864872 22147 leader_election.cc:302] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c83614a70cdc4346b70f189d08530d87, fb299619d2254711bbb9f2d794e2e6be; no voters:
I20260924 21:16:52.864996 22569 raft_consensus.cc:2781] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:52.865051 22569 raft_consensus.cc:482] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:52.865077 22569 raft_consensus.cc:3037] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:52.865733 22569 raft_consensus.cc:504] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.865908 22569 leader_election.cc:288] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 election: Requested vote from peers c83614a70cdc4346b70f189d08530d87 (127.20.115.194:37579), 7a944235b6bd487e97691275d9e144bb (127.20.115.195:40211)
I20260924 21:16:52.866084 22355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "649369e5fda54bc78a6723132f47ab16" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83614a70cdc4346b70f189d08530d87"
I20260924 21:16:52.866140 22495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "649369e5fda54bc78a6723132f47ab16" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a944235b6bd487e97691275d9e144bb"
I20260924 21:16:52.866191 22355 raft_consensus.cc:3037] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:52.866214 22495 raft_consensus.cc:3037] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:52.866883 22355 raft_consensus.cc:2454] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 1.
I20260924 21:16:52.866935 22495 raft_consensus.cc:2454] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 1.
I20260924 21:16:52.867115 22147 leader_election.cc:302] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a944235b6bd487e97691275d9e144bb, fb299619d2254711bbb9f2d794e2e6be; no voters:
I20260924 21:16:52.867213 22569 raft_consensus.cc:2781] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:52.867336 22569 raft_consensus.cc:686] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 1 LEADER]: Becoming Leader. State: Replica: fb299619d2254711bbb9f2d794e2e6be, State: Running, Role: LEADER
I20260924 21:16:52.867430 22569 consensus_queue.cc:237] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.868217 22057 catalog_manager.cc:5495] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be reported cstate change: term changed from 0 to 1, leader changed from <none> to fb299619d2254711bbb9f2d794e2e6be (127.20.115.193). New cstate: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } health_report { overall_health: UNKNOWN } } }
I20260924 21:16:52.942812 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:52.940441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52402 (local address 127.20.115.193:46627)
0924 21:16:52.940510 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:52.940511 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:52.940519 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:52.940553 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:52.940554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:52.940572 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:52.940637 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:52.940821 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.941317 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.942071 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:52.942248 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:52.942298 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:52.942536 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:52.942622 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:52.942678 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:52.942738 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 21:16:52.942755 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:16:52.944574 22495 raft_consensus.cc:1261] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Refusing update from remote peer fb299619d2254711bbb9f2d794e2e6be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:52.944598 22355 raft_consensus.cc:1261] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Refusing update from remote peer fb299619d2254711bbb9f2d794e2e6be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:52.945006 22575 consensus_queue.cc:1035] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:52.945159 22569 consensus_queue.cc:1035] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:52.947579 22580 mvcc.cc:205] Tried to move back new op lower bound from 7333005774618238976 to 7333005774305660928. Current Snapshot: MvccSnapshot[applied={T|T < 7333005774618238976}]
I20260924 21:16:52.947916 22578 mvcc.cc:205] Tried to move back new op lower bound from 7333005774618238976 to 7333005774305660928. Current Snapshot: MvccSnapshot[applied={T|T < 7333005774618238976}]
I20260924 21:16:52.951248 22058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:52.951327 22058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:52.952039 22058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3: Sending DeleteTablet for 3 replicas of tablet 649369e5fda54bc78a6723132f47ab16
I20260924 21:16:52.952356 22195 tablet_service.cc:1514] Processing DeleteTablet for tablet 649369e5fda54bc78a6723132f47ab16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:52 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:52398
I20260924 21:16:52.952425 22461 tablet_service.cc:1514] Processing DeleteTablet for tablet 649369e5fda54bc78a6723132f47ab16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:52 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:55340
I20260924 21:16:52.952569 22335 tablet_service.cc:1514] Processing DeleteTablet for tablet 649369e5fda54bc78a6723132f47ab16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:52 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:34144
I20260924 21:16:52.952750 22588 tablet_replica.cc:330] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: stopping tablet replica
I20260924 21:16:52.952893 22588 raft_consensus.cc:2229] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:52.952905 22590 tablet_replica.cc:330] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: stopping tablet replica
I20260924 21:16:52.952996 22590 raft_consensus.cc:2229] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:52.952977 22058 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 21:16:52.953080 22588 raft_consensus.cc:2258] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 21:16:52.953125 22058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:52.952829 22589 tablet_replica.cc:330] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: stopping tablet replica
I20260924 21:16:52.953404 22590 raft_consensus.cc:2258] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:52.953439 22589 raft_consensus.cc:2229] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:52.953625 22589 raft_consensus.cc:2258] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:52.953706 22588 ts_tablet_manager.cc:1905] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:52.954077 22589 ts_tablet_manager.cc:1905] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:52.954875 22195 tablet_service.cc:1467] Processing CreateTablet for tablet 045b500555c4433ba27559be6d9cbf2c (DEFAULT_TABLE table=test-table [id=bb96e8ae417c4fd789d3c1fe224ccbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:52.954983 22195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 045b500555c4433ba27559be6d9cbf2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:52.955696 22589 ts_tablet_manager.cc:1918] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:52.955700 22588 ts_tablet_manager.cc:1918] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:52.955757 22588 log.cc:1196] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/649369e5fda54bc78a6723132f47ab16
I20260924 21:16:52.955757 22589 log.cc:1196] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/649369e5fda54bc78a6723132f47ab16
I20260924 21:16:52.956015 22589 ts_tablet_manager.cc:1939] T 649369e5fda54bc78a6723132f47ab16 P 7a944235b6bd487e97691275d9e144bb: Deleting consensus metadata
I20260924 21:16:52.956018 22588 ts_tablet_manager.cc:1939] T 649369e5fda54bc78a6723132f47ab16 P fb299619d2254711bbb9f2d794e2e6be: Deleting consensus metadata
I20260924 21:16:52.956260 22335 tablet_service.cc:1467] Processing CreateTablet for tablet 045b500555c4433ba27559be6d9cbf2c (DEFAULT_TABLE table=test-table [id=bb96e8ae417c4fd789d3c1fe224ccbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:52.956355 22335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 045b500555c4433ba27559be6d9cbf2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:52.956903 22565 tablet_bootstrap.cc:492] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: Bootstrap starting.
I20260924 21:16:52.957295 22565 tablet_bootstrap.cc:654] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:52.957367 22570 tablet_bootstrap.cc:492] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: Bootstrap starting.
I20260924 21:16:52.957631 22461 tablet_service.cc:1467] Processing CreateTablet for tablet 045b500555c4433ba27559be6d9cbf2c (DEFAULT_TABLE table=test-table [id=bb96e8ae417c4fd789d3c1fe224ccbb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:52.957717 22461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 045b500555c4433ba27559be6d9cbf2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:52.957741 22570 tablet_bootstrap.cc:654] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:52.957918 22565 tablet_bootstrap.cc:492] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: No bootstrap required, opened a new log
I20260924 21:16:52.957958 22565 ts_tablet_manager.cc:1397] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:52.958092 22565 raft_consensus.cc:348] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.958145 22565 raft_consensus.cc:374] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:52.958158 22565 raft_consensus.cc:729] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb299619d2254711bbb9f2d794e2e6be, State: Initialized, Role: FOLLOWER
I20260924 21:16:52.958217 22565 consensus_queue.cc:260] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.958302 22565 ts_tablet_manager.cc:1428] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.958693 22570 tablet_bootstrap.cc:492] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: No bootstrap required, opened a new log
I20260924 21:16:52.958724 22570 ts_tablet_manager.cc:1397] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:52.958886 22564 tablet_bootstrap.cc:492] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: Bootstrap starting.
I20260924 21:16:52.958866 22570 raft_consensus.cc:348] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.958974 22570 raft_consensus.cc:374] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:52.958998 22570 raft_consensus.cc:729] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c83614a70cdc4346b70f189d08530d87, State: Initialized, Role: FOLLOWER
I20260924 21:16:52.959028 22043 catalog_manager.cc:4841] TS 7a944235b6bd487e97691275d9e144bb (127.20.115.195:40211): tablet 649369e5fda54bc78a6723132f47ab16 (table test-table [id=4be45164eeb14271a18f80210e346748]) successfully deleted
I20260924 21:16:52.959051 22570 consensus_queue.cc:260] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.959151 22570 ts_tablet_manager.cc:1428] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:52.959251 22042 catalog_manager.cc:4841] TS fb299619d2254711bbb9f2d794e2e6be (127.20.115.193:46627): tablet 649369e5fda54bc78a6723132f47ab16 (table test-table [id=4be45164eeb14271a18f80210e346748]) successfully deleted
I20260924 21:16:52.959300 22564 tablet_bootstrap.cc:654] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:52.960290 22590 ts_tablet_manager.cc:1905] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:52.961120 22564 tablet_bootstrap.cc:492] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: No bootstrap required, opened a new log
I20260924 21:16:52.961185 22564 ts_tablet_manager.cc:1397] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:16:52.961325 22564 raft_consensus.cc:348] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.961369 22564 raft_consensus.cc:374] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:52.961381 22564 raft_consensus.cc:729] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a944235b6bd487e97691275d9e144bb, State: Initialized, Role: FOLLOWER
I20260924 21:16:52.961433 22564 consensus_queue.cc:260] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:52.961517 22564 ts_tablet_manager.cc:1428] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:52.961750 22590 ts_tablet_manager.cc:1918] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:16:52.961858 22590 log.cc:1196] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/649369e5fda54bc78a6723132f47ab16
I20260924 21:16:52.962584 22590 ts_tablet_manager.cc:1939] T 649369e5fda54bc78a6723132f47ab16 P c83614a70cdc4346b70f189d08530d87: Deleting consensus metadata
I20260924 21:16:52.962939 22043 catalog_manager.cc:4841] TS c83614a70cdc4346b70f189d08530d87 (127.20.115.194:37579): tablet 649369e5fda54bc78a6723132f47ab16 (table test-table [id=4be45164eeb14271a18f80210e346748]) successfully deleted
I20260924 21:16:53.126362 22583 raft_consensus.cc:482] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:53.126471 22583 raft_consensus.cc:504] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.126605 22583 leader_election.cc:288] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c83614a70cdc4346b70f189d08530d87 (127.20.115.194:37579), 7a944235b6bd487e97691275d9e144bb (127.20.115.195:40211)
I20260924 21:16:53.126861 22355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "045b500555c4433ba27559be6d9cbf2c" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83614a70cdc4346b70f189d08530d87" is_pre_election: true
I20260924 21:16:53.126926 22495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "045b500555c4433ba27559be6d9cbf2c" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a944235b6bd487e97691275d9e144bb" is_pre_election: true
I20260924 21:16:53.127008 22355 raft_consensus.cc:2454] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 0.
I20260924 21:16:53.127038 22495 raft_consensus.cc:2454] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 0.
I20260924 21:16:53.127207 22147 leader_election.cc:302] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a944235b6bd487e97691275d9e144bb, fb299619d2254711bbb9f2d794e2e6be; no voters:
I20260924 21:16:53.127318 22583 raft_consensus.cc:2781] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:53.127372 22583 raft_consensus.cc:482] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:53.127393 22583 raft_consensus.cc:3037] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:53.128111 22583 raft_consensus.cc:504] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.128252 22583 leader_election.cc:288] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 election: Requested vote from peers c83614a70cdc4346b70f189d08530d87 (127.20.115.194:37579), 7a944235b6bd487e97691275d9e144bb (127.20.115.195:40211)
I20260924 21:16:53.128436 22355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "045b500555c4433ba27559be6d9cbf2c" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83614a70cdc4346b70f189d08530d87"
I20260924 21:16:53.128506 22355 raft_consensus.cc:3037] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:53.128501 22495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "045b500555c4433ba27559be6d9cbf2c" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a944235b6bd487e97691275d9e144bb"
I20260924 21:16:53.128619 22495 raft_consensus.cc:3037] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:53.129268 22355 raft_consensus.cc:2454] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 1.
I20260924 21:16:53.129474 22147 leader_election.cc:302] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c83614a70cdc4346b70f189d08530d87, fb299619d2254711bbb9f2d794e2e6be; no voters:
I20260924 21:16:53.129585 22583 raft_consensus.cc:2781] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:53.129627 22583 raft_consensus.cc:686] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 1 LEADER]: Becoming Leader. State: Replica: fb299619d2254711bbb9f2d794e2e6be, State: Running, Role: LEADER
I20260924 21:16:53.129689 22495 raft_consensus.cc:2454] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 1.
I20260924 21:16:53.129679 22583 consensus_queue.cc:237] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.130352 22057 catalog_manager.cc:5495] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be reported cstate change: term changed from 0 to 1, leader changed from <none> to fb299619d2254711bbb9f2d794e2e6be (127.20.115.193). New cstate: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } health_report { overall_health: UNKNOWN } } }
Sep 24 21:16:53 dist-test-slave-cxpd krb5kdc[22013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284611, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:53.168970 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:53.161595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45461 (local address 127.0.0.1:53312)
0924 21:16:53.161878 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:53.161890 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:53.161958 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:53.162231 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:53.162232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:53.162552 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:53.162602 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:53.162604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.163334 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.163335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:53.163911 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:53.163912 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.164281 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.164282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:53.164331 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:53.165076 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:53.165081 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:53.166437 (+ 1356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:53.167960 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:53.167966 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:53.167969 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:53.168178 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:53.168357 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:53.168358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:53.168358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:53.168413 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:53.168630 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:53.168636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:53.168677 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 21:16:53.168728 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:16:53.168895 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":83,"threads_started":1}
I20260924 21:16:53.169109 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:53.161646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53312 (local address 127.20.115.254:45461)
0924 21:16:53.161787 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:53.161789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:53.161955 (+ 166us) server_negotiation.cc:407] Connection header received
0924 21:16:53.162126 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:53.162127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:53.162137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:53.162213 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:53.162684 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.163254 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.163986 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.164191 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.164245 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:53.166548 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:53.166549 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:53.166549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:53.166555 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:53.167882 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:53.168244 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:53.168245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:53.168245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:53.168299 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:53.168466 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:53.168466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:53.168467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:53.168576 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:53.168628 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:53.168761 (+ 133us) server_negotiation.cc:299] Negotiation successful
0924 21:16:53.168962 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":107,"thread_start_us":85,"threads_started":1}
I20260924 21:16:53.171131 22057 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:53312:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 21:16:53.171370 22057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:53.172919 22195 tablet_service.cc:1467] Processing CreateTablet for tablet 627a54da92c34e71b9ffcf221ad82f2f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=95d60e034ce14a13b6fcc698a1613eb5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:16:53.172955 22335 tablet_service.cc:1467] Processing CreateTablet for tablet 627a54da92c34e71b9ffcf221ad82f2f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=95d60e034ce14a13b6fcc698a1613eb5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:16:53.172971 22461 tablet_service.cc:1467] Processing CreateTablet for tablet 627a54da92c34e71b9ffcf221ad82f2f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=95d60e034ce14a13b6fcc698a1613eb5]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:16:53.173099 22461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627a54da92c34e71b9ffcf221ad82f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:53.173091 22335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627a54da92c34e71b9ffcf221ad82f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:53.173092 22195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627a54da92c34e71b9ffcf221ad82f2f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:53.174314 22564 tablet_bootstrap.cc:492] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: Bootstrap starting.
I20260924 21:16:53.174543 22565 tablet_bootstrap.cc:492] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: Bootstrap starting.
I20260924 21:16:53.174533 22570 tablet_bootstrap.cc:492] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: Bootstrap starting.
I20260924 21:16:53.174765 22564 tablet_bootstrap.cc:654] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:53.174844 22565 tablet_bootstrap.cc:654] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:53.175028 22570 tablet_bootstrap.cc:654] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:53.175405 22565 tablet_bootstrap.cc:492] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: No bootstrap required, opened a new log
I20260924 21:16:53.175464 22565 ts_tablet_manager.cc:1397] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:53.175597 22565 raft_consensus.cc:348] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.175671 22565 raft_consensus.cc:374] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:53.175693 22565 raft_consensus.cc:729] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb299619d2254711bbb9f2d794e2e6be, State: Initialized, Role: FOLLOWER
I20260924 21:16:53.175734 22564 tablet_bootstrap.cc:492] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: No bootstrap required, opened a new log
I20260924 21:16:53.175779 22564 ts_tablet_manager.cc:1397] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:53.175768 22565 consensus_queue.cc:260] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.175800 22570 tablet_bootstrap.cc:492] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: No bootstrap required, opened a new log
I20260924 21:16:53.175843 22570 ts_tablet_manager.cc:1397] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:53.175863 22565 ts_tablet_manager.cc:1428] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:53.175877 22569 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.175946 22569 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:53.175932 22564 raft_consensus.cc:348] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.176010 22564 raft_consensus.cc:374] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:53.175982 22570 raft_consensus.cc:348] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.176031 22564 raft_consensus.cc:729] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a944235b6bd487e97691275d9e144bb, State: Initialized, Role: FOLLOWER
I20260924 21:16:53.176048 22570 raft_consensus.cc:374] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:53.176072 22570 raft_consensus.cc:729] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c83614a70cdc4346b70f189d08530d87, State: Initialized, Role: FOLLOWER
I20260924 21:16:53.176077 22564 consensus_queue.cc:260] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.176115 22570 consensus_queue.cc:260] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.176173 22564 ts_tablet_manager.cc:1428] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:53.176210 22570 ts_tablet_manager.cc:1428] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:53.176440 22568 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.176554 22568 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:53.176690 22572 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.176769 22572 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:53.269325 22569 raft_consensus.cc:482] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:53.269423 22569 raft_consensus.cc:504] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.269567 22569 leader_election.cc:288] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c83614a70cdc4346b70f189d08530d87 (127.20.115.194:37579), 7a944235b6bd487e97691275d9e144bb (127.20.115.195:40211)
I20260924 21:16:53.269781 22355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627a54da92c34e71b9ffcf221ad82f2f" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83614a70cdc4346b70f189d08530d87" is_pre_election: true
I20260924 21:16:53.269816 22495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627a54da92c34e71b9ffcf221ad82f2f" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a944235b6bd487e97691275d9e144bb" is_pre_election: true
I20260924 21:16:53.269886 22355 raft_consensus.cc:2454] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 0.
I20260924 21:16:53.269922 22495 raft_consensus.cc:2454] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 0.
I20260924 21:16:53.270040 22147 leader_election.cc:302] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c83614a70cdc4346b70f189d08530d87, fb299619d2254711bbb9f2d794e2e6be; no voters:
I20260924 21:16:53.270179 22569 raft_consensus.cc:2781] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:53.270224 22569 raft_consensus.cc:482] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:53.270248 22569 raft_consensus.cc:3037] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:53.270916 22569 raft_consensus.cc:504] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.271103 22569 leader_election.cc:288] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 election: Requested vote from peers c83614a70cdc4346b70f189d08530d87 (127.20.115.194:37579), 7a944235b6bd487e97691275d9e144bb (127.20.115.195:40211)
I20260924 21:16:53.271283 22355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627a54da92c34e71b9ffcf221ad82f2f" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c83614a70cdc4346b70f189d08530d87"
I20260924 21:16:53.271307 22495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627a54da92c34e71b9ffcf221ad82f2f" candidate_uuid: "fb299619d2254711bbb9f2d794e2e6be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a944235b6bd487e97691275d9e144bb"
I20260924 21:16:53.271369 22495 raft_consensus.cc:3037] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:53.271369 22355 raft_consensus.cc:3037] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:53.272099 22355 raft_consensus.cc:2454] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 1.
I20260924 21:16:53.272099 22495 raft_consensus.cc:2454] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb299619d2254711bbb9f2d794e2e6be in term 1.
I20260924 21:16:53.272333 22147 leader_election.cc:302] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a944235b6bd487e97691275d9e144bb, fb299619d2254711bbb9f2d794e2e6be; no voters:
I20260924 21:16:53.272447 22569 raft_consensus.cc:2781] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:53.272544 22569 raft_consensus.cc:686] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [term 1 LEADER]: Becoming Leader. State: Replica: fb299619d2254711bbb9f2d794e2e6be, State: Running, Role: LEADER
I20260924 21:16:53.272604 22569 consensus_queue.cc:237] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } }
I20260924 21:16:53.273005 22583 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: TxnStatusTablet state changed. Reason: New leader fb299619d2254711bbb9f2d794e2e6be. Latest consensus state: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.273099 22583 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:16:53.273293 22606 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 21:16:53.273432 22606 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 21:16:53.273365 22057 catalog_manager.cc:5495] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be reported cstate change: term changed from 0 to 1, leader changed from <none> to fb299619d2254711bbb9f2d794e2e6be (127.20.115.193). New cstate: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } health_report { overall_health: UNKNOWN } } }
I20260924 21:16:53.294363 22600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:53.291670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:46627 (local address 127.0.0.1:52410)
0924 21:16:53.291749 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:53.291762 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:53.291815 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:53.291963 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:53.291964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:53.291965 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:16:53.292010 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:53.292012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.292917 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.292919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:53.293512 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:53.293514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.293984 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.293984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:53.294027 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:53.294042 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:16:53.294221 (+ 179us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:16:53.294227 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 21:16:53.294266 (+ 39us) client_negotiation.cc:240] Negotiation successful
0924 21:16:53.294304 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 21:16:53.294374 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:53.291751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52410 (local address 127.20.115.193:46627)
0924 21:16:53.291805 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:53.291807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:53.291818 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:53.291884 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:53.291885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:53.291897 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:53.291959 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:53.292049 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.292815 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.293603 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.293903 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.293974 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:53.294121 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:53.294157 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:53.294209 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:53.294269 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 21:16:53.294291 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:16:53.295426 22355 raft_consensus.cc:1261] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Refusing update from remote peer fb299619d2254711bbb9f2d794e2e6be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:53.295426 22495 raft_consensus.cc:1261] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Refusing update from remote peer fb299619d2254711bbb9f2d794e2e6be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:53.295643 22583 consensus_queue.cc:1035] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:53.295746 22569 consensus_queue.cc:1035] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:53.296221 22572 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: TxnStatusTablet state changed. Reason: New leader fb299619d2254711bbb9f2d794e2e6be. Latest consensus state: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.296319 22572 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:53.296388 22568 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: TxnStatusTablet state changed. Reason: New leader fb299619d2254711bbb9f2d794e2e6be. Latest consensus state: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.296469 22568 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:53.296844 22569 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.296927 22569 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:16:53.297297 22575 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.297369 22575 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P fb299619d2254711bbb9f2d794e2e6be: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:16:53.297681 22579 mvcc.cc:205] Tried to move back new op lower bound from 7333005776056397824 to 7333005775965114368. Current Snapshot: MvccSnapshot[applied={T|T < 7333005776056397824}]
I20260924 21:16:53.297767 22572 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.297855 22572 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P c83614a70cdc4346b70f189d08530d87: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:53.298776 22568 tablet_replica.cc:439] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb299619d2254711bbb9f2d794e2e6be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb299619d2254711bbb9f2d794e2e6be" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 46627 } } peers { permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 } } peers { permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 } } }
I20260924 21:16:53.298858 22568 tablet_replica.cc:442] T 627a54da92c34e71b9ffcf221ad82f2f P 7a944235b6bd487e97691275d9e144bb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:53.307135 22613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:53.301070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:46627 (local address 127.20.115.193:42913)
0924 21:16:53.304242 (+ 3172us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:53.304255 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:53.304325 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:53.304540 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:53.304541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:53.304543 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:16:53.304589 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:53.304590 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.305383 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.305384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:53.306316 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:53.306318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.306676 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.306677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:53.306735 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:53.306750 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:16:53.306935 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:16:53.306936 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:16:53.306987 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:16:53.307030 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3130,"thread_start_us":78,"threads_started":1}
I20260924 21:16:53.307143 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:53.301132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:42913 (local address 127.20.115.193:46627)
0924 21:16:53.301168 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:53.301169 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:53.304417 (+ 3248us) server_negotiation.cc:407] Connection header received
0924 21:16:53.304447 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:53.304448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:53.304458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:53.304517 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:53.304677 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.305283 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.306414 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.306602 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.306657 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:53.306842 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:53.306870 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:53.306918 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:53.306986 (+ 68us) server_negotiation.cc:299] Negotiation successful
0924 21:16:53.307007 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":16}
I20260924 21:16:53.309271 22495 raft_consensus.cc:1261] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Refusing update from remote peer fb299619d2254711bbb9f2d794e2e6be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:53.309310 22355 raft_consensus.cc:1261] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Refusing update from remote peer fb299619d2254711bbb9f2d794e2e6be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:53.309492 22575 consensus_queue.cc:1035] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "c83614a70cdc4346b70f189d08530d87" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:53.309574 22583 consensus_queue.cc:1035] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a944235b6bd487e97691275d9e144bb" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 40211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:53.320156 22057 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:53.320263 22057 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:53.320919 22057 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 890ea9271891423583054c1d3330aae3: Sending DeleteTablet for 3 replicas of tablet 045b500555c4433ba27559be6d9cbf2c
I20260924 21:16:53.321185 22335 tablet_service.cc:1514] Processing DeleteTablet for tablet 045b500555c4433ba27559be6d9cbf2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:53 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:34144
I20260924 21:16:53.321215 22194 tablet_service.cc:1514] Processing DeleteTablet for tablet 045b500555c4433ba27559be6d9cbf2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:53 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:52398
I20260924 21:16:53.321214 22461 tablet_service.cc:1514] Processing DeleteTablet for tablet 045b500555c4433ba27559be6d9cbf2c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:53 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:55340
I20260924 21:16:53.321280 22590 tablet_replica.cc:330] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: stopping tablet replica
I20260924 21:16:53.321302 22588 tablet_replica.cc:330] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: stopping tablet replica
I20260924 21:16:53.321333 22590 raft_consensus.cc:2229] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:53.321354 22588 raft_consensus.cc:2229] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:53.321370 22590 raft_consensus.cc:2258] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:53.321406 22588 raft_consensus.cc:2258] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:53.321441 22589 tablet_replica.cc:330] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: stopping tablet replica
I20260924 21:16:53.321491 22589 raft_consensus.cc:2229] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:53.321535 22589 raft_consensus.cc:2258] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:53.321731 22590 ts_tablet_manager.cc:1905] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:53.322058 22589 ts_tablet_manager.cc:1905] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:53.322156 22588 ts_tablet_manager.cc:1905] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:53.323405 22590 ts_tablet_manager.cc:1918] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:16:53.323458 22590 log.cc:1196] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/045b500555c4433ba27559be6d9cbf2c
I20260924 21:16:53.323688 22590 ts_tablet_manager.cc:1939] T 045b500555c4433ba27559be6d9cbf2c P c83614a70cdc4346b70f189d08530d87: Deleting consensus metadata
I20260924 21:16:53.323698 22588 ts_tablet_manager.cc:1918] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:16:53.323747 22588 log.cc:1196] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/045b500555c4433ba27559be6d9cbf2c
I20260924 21:16:53.323772 22589 ts_tablet_manager.cc:1918] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:16:53.323820 22589 log.cc:1196] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/045b500555c4433ba27559be6d9cbf2c
I20260924 21:16:53.323972 22588 ts_tablet_manager.cc:1939] T 045b500555c4433ba27559be6d9cbf2c P fb299619d2254711bbb9f2d794e2e6be: Deleting consensus metadata
I20260924 21:16:53.324028 22589 ts_tablet_manager.cc:1939] T 045b500555c4433ba27559be6d9cbf2c P 7a944235b6bd487e97691275d9e144bb: Deleting consensus metadata
I20260924 21:16:53.324195 22043 catalog_manager.cc:4841] TS c83614a70cdc4346b70f189d08530d87 (127.20.115.194:37579): tablet 045b500555c4433ba27559be6d9cbf2c (table test-table [id=bb96e8ae417c4fd789d3c1fe224ccbb3]) successfully deleted
I20260924 21:16:53.324503 22043 catalog_manager.cc:4841] TS 7a944235b6bd487e97691275d9e144bb (127.20.115.195:40211): tablet 045b500555c4433ba27559be6d9cbf2c (table test-table [id=bb96e8ae417c4fd789d3c1fe224ccbb3]) successfully deleted
I20260924 21:16:53.324523 22042 catalog_manager.cc:4841] TS fb299619d2254711bbb9f2d794e2e6be (127.20.115.193:46627): tablet 045b500555c4433ba27559be6d9cbf2c (table test-table [id=bb96e8ae417c4fd789d3c1fe224ccbb3]) successfully deleted
Sep 24 21:16:53 dist-test-slave-cxpd krb5kdc[22013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284612, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
I20260924 21:16:53.333225 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:53.324626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52420 (local address 127.20.115.193:46627)
0924 21:16:53.324669 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:53.324671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:53.324979 (+ 308us) server_negotiation.cc:407] Connection header received
0924 21:16:53.325010 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:53.325011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:53.325021 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:53.325090 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:53.325589 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.326320 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.327005 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.327301 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.327371 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:53.330418 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:53.330427 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:53.330427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:53.330433 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:53.332028 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:53.332419 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:53.332422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:53.332422 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:53.332474 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:53.332662 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:53.332663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:53.332663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:53.332837 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:53.332958 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:53.333077 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 21:16:53.333142 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:16:53.333484 22255 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52420
I20260924 21:16:53.340404 22601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:53.334410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.20.115.254:45461)
0924 21:16:53.334462 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:53.334464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:53.334473 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:16:53.334538 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:53.334539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:53.334550 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:53.334604 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:53.335100 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.335594 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.336275 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:53.336493 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:53.336572 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:53.337722 (+ 1150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:53.337723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:53.337724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:53.337731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:53.339193 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:53.339550 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:53.339551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:53.339552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:53.339608 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:53.339838 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:53.339839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:53.339839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:53.339961 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:53.340078 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:53.340213 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 21:16:53.340300 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260924 21:16:53.340682 22057 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53316
I20260924 21:16:53.341387 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 22130
I20260924 21:16:53.348855 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 22270
I20260924 21:16:53.353601 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 22410
I20260924 21:16:53.358377 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 22026
2026-09-24T21:16:53Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2608 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:53 dist-test-slave-cxpd krb5kdc[22623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:53 dist-test-slave-cxpd krb5kdc[22623](info): set up 2 sockets
Sep 24 21:16:53 dist-test-slave-cxpd krb5kdc[22623](info): commencing operation
krb5kdc: starting...
W20260924 21:16:54.392400 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:16:54 dist-test-slave-cxpd krb5kdc[22623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:16:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:16:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:16:54.512240 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:39003
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:39003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:54.592136 22636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:54.592396 22636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:54.592430 22636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:54.593660 22636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:16:54.593730 22636 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 21:16:54.593755 22636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:54.593777 22636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:54.593811 22636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:16:54.593833 22636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:16:54.595323 22636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:39003
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:39003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:54.595598 22636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:54.595827 22636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:54.600682 22642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:54.600695 22641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:54.600698 22644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:54.600730 22636 server_base.cc:1034] running on GCE node
I20260924 21:16:54.601059 22636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:54.601352 22636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:54.602504 22636 hybrid_clock.cc:648] HybridClock initialized: now 1790284614602493 us; error 34 us; skew 500 ppm
Sep 24 21:16:54 dist-test-slave-cxpd krb5kdc[22623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284614, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:54.605041 22636 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:16:54.605501 22636 webserver.cc:460] Webserver started at http://127.20.115.254:38631/ using document root <none> and password file <none>
I20260924 21:16:54.605693 22636 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:54.605733 22636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:54.605822 22636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:54.606818 22636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "888d0733c28a4d68a90d09b03c73dc47"
format_stamp: "Formatted at 2026-09-24 21:16:54 on dist-test-slave-cxpd"
server_key: "c2e76106fea0fda8c13dca002b04f0bc"
server_key_iv: "ed8a861703c7a4ce47732e40b6a40df6"
server_key_version: "encryptionkey@0"
I20260924 21:16:54.607126 22636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "888d0733c28a4d68a90d09b03c73dc47"
format_stamp: "Formatted at 2026-09-24 21:16:54 on dist-test-slave-cxpd"
server_key: "c2e76106fea0fda8c13dca002b04f0bc"
server_key_iv: "ed8a861703c7a4ce47732e40b6a40df6"
server_key_version: "encryptionkey@0"
I20260924 21:16:54.608372 22636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 21:16:54.609165 22651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:54.609355 22636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:54.609462 22636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "888d0733c28a4d68a90d09b03c73dc47"
format_stamp: "Formatted at 2026-09-24 21:16:54 on dist-test-slave-cxpd"
server_key: "c2e76106fea0fda8c13dca002b04f0bc"
server_key_iv: "ed8a861703c7a4ce47732e40b6a40df6"
server_key_version: "encryptionkey@0"
I20260924 21:16:54.609550 22636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:54.627460 22636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:54.629477 22636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:54.629653 22636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:54.634583 22636 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:39003
I20260924 21:16:54.634641 22713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:39003 every 8 connection(s)
I20260924 21:16:54.635102 22636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:16:54.635725 22714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:54.636965 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 22636
I20260924 21:16:54.637063 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:16:54.637228 20943 external_mini_cluster.cc:1428] Setting key e8cd4b2cd48ad782eb17e02a012eda96
I20260924 21:16:54.638415 22714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47: Bootstrap starting.
I20260924 21:16:54.638990 22714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:54.639263 22714 log.cc:824] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:54.639945 22714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47: No bootstrap required, opened a new log
I20260924 21:16:54.641762 22714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d0733c28a4d68a90d09b03c73dc47" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39003 } }
I20260924 21:16:54.641924 22714 raft_consensus.cc:374] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:54.641971 22714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 888d0733c28a4d68a90d09b03c73dc47, State: Initialized, Role: FOLLOWER
I20260924 21:16:54.642130 22714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d0733c28a4d68a90d09b03c73dc47" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39003 } }
I20260924 21:16:54.642241 22714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:16:54.642287 22714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:16:54.642366 22714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:54.642879 22714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d0733c28a4d68a90d09b03c73dc47" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39003 } }
I20260924 21:16:54.642995 22714 leader_election.cc:302] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 888d0733c28a4d68a90d09b03c73dc47; no voters:
I20260924 21:16:54.643174 22714 leader_election.cc:288] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:16:54.643198 22719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:54.643311 22719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [term 1 LEADER]: Becoming Leader. State: Replica: 888d0733c28a4d68a90d09b03c73dc47, State: Running, Role: LEADER
I20260924 21:16:54.643385 22719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d0733c28a4d68a90d09b03c73dc47" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39003 } }
I20260924 21:16:54.643437 22714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:16:54.643676 22719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "888d0733c28a4d68a90d09b03c73dc47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d0733c28a4d68a90d09b03c73dc47" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39003 } } }
I20260924 21:16:54.643731 22719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [sys.catalog]: This master's current role is: LEADER
I20260924 21:16:54.643836 22719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 888d0733c28a4d68a90d09b03c73dc47. Latest consensus state: current_term: 1 leader_uuid: "888d0733c28a4d68a90d09b03c73dc47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "888d0733c28a4d68a90d09b03c73dc47" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39003 } } }
I20260924 21:16:54.643889 22719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47 [sys.catalog]: This master's current role is: LEADER
Sep 24 21:16:54 dist-test-slave-cxpd krb5kdc[22623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284614, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:54.644675 22723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:16:54.645283 22723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:16:54.647259 22723 catalog_manager.cc:1347] Generated new cluster ID: be7e4fcaa3cb4cceac61ac2efb279afd
I20260924 21:16:54.647351 22723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:16:54.648195 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:54.637794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54728 (local address 127.20.115.254:39003)
0924 21:16:54.637976 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:54.637993 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:54.638020 (+ 27us) server_negotiation.cc:407] Connection header received
0924 21:16:54.638408 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:54.638414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:54.638458 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:54.638653 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:54.639330 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.640439 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.641277 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.641689 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.641809 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:54.644494 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:54.644498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:54.644499 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:54.644512 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:54.646445 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:54.646829 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:54.646832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:54.646832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:54.646951 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:54.647213 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:54.647214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:54.647215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:54.647515 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:54.647660 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:54.647842 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 21:16:54.647970 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
I20260924 21:16:54.662014 22723 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:16:54.662720 22723 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:16:54.670068 22723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47: Generated new TSK 0
I20260924 21:16:54.670286 22723 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:16:54.704190 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:39003
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:54.785534 22740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:54.785741 22740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:54.785765 22740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:54.786964 22740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:54.787007 22740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:54.787029 22740 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 21:16:54.787039 22740 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:16:54.787057 22740 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:16:54.788514 22740 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.115.254:39003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:54.788762 22740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:54.789031 22740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:54.794613 22746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:54.794585 22748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:54.794632 22740 server_base.cc:1034] running on GCE node
W20260924 21:16:54.794579 22745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:54.795044 22740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:54.795288 22740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:54.796492 22740 hybrid_clock.cc:648] HybridClock initialized: now 1790284614796477 us; error 29 us; skew 500 ppm
Sep 24 21:16:54 dist-test-slave-cxpd krb5kdc[22623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284614, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:54.799132 22740 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:16:54.799659 22740 webserver.cc:460] Webserver started at http://127.20.115.193:36461/ using document root <none> and password file <none>
I20260924 21:16:54.799857 22740 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:54.799897 22740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:54.799988 22740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:54.801074 22740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "50c42c83fd7f4af29c1bdf3db5b81403"
format_stamp: "Formatted at 2026-09-24 21:16:54 on dist-test-slave-cxpd"
server_key: "af442ea710dc94b7e14d666932699dd6"
server_key_iv: "8cbe941f1c67518d4581554c9e4ff400"
server_key_version: "encryptionkey@0"
I20260924 21:16:54.801371 22740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "50c42c83fd7f4af29c1bdf3db5b81403"
format_stamp: "Formatted at 2026-09-24 21:16:54 on dist-test-slave-cxpd"
server_key: "af442ea710dc94b7e14d666932699dd6"
server_key_iv: "8cbe941f1c67518d4581554c9e4ff400"
server_key_version: "encryptionkey@0"
I20260924 21:16:54.802624 22740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:54.803356 22755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:54.803563 22740 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:54.803648 22740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "50c42c83fd7f4af29c1bdf3db5b81403"
format_stamp: "Formatted at 2026-09-24 21:16:54 on dist-test-slave-cxpd"
server_key: "af442ea710dc94b7e14d666932699dd6"
server_key_iv: "8cbe941f1c67518d4581554c9e4ff400"
server_key_version: "encryptionkey@0"
I20260924 21:16:54.803702 22740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:54.839198 22740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:54.841544 22740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:54.841703 22740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:54.842273 22740 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:54.842312 22740 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:54.842338 22740 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:54.842353 22740 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:54.849454 22740 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:43673
I20260924 21:16:54.849546 22870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:43673 every 8 connection(s)
I20260924 21:16:54.849903 22740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:16:54 dist-test-slave-cxpd krb5kdc[22623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284614, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:54.850376 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 22740
I20260924 21:16:54.850476 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:16:54.850652 20943 external_mini_cluster.cc:1428] Setting key 856e048d3af6be9dcb674c431843b7fc
I20260924 21:16:54.854228 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:54.842985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:47711 (local address 127.20.115.254:39003)
0924 21:16:54.843051 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:54.843054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:54.843288 (+ 234us) server_negotiation.cc:407] Connection header received
0924 21:16:54.843940 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:54.843942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:54.843954 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:54.844013 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:54.844964 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.845795 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.847549 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.847764 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.847833 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:54.851316 (+ 3483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:54.851327 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:54.851327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:54.851336 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:54.852733 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:54.853213 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:54.853215 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:54.853215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:54.853286 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:54.853557 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:54.853558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:54.853558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:54.853680 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:54.853760 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:54.854037 (+ 277us) server_negotiation.cc:299] Negotiation successful
0924 21:16:54.854137 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:16:54.854426 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:54.842946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39003 (local address 127.20.115.193:47711)
0924 21:16:54.843135 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:54.843189 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:54.843801 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:54.844091 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:54.844097 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:54.844479 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:54.844806 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:54.844813 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.845915 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.845917 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:54.847443 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:54.847446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.847889 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.847891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:54.848061 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:54.848708 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:54.848719 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:54.851219 (+ 2500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:54.852875 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:54.852881 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:54.852883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:54.853130 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:54.853354 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:54.853355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:54.853356 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:54.853472 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:54.853751 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:54.853757 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:54.853903 (+ 146us) client_negotiation.cc:769] Sending connection context
0924 21:16:54.854016 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:16:54.854185 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":105,"threads_started":1}
I20260924 21:16:54.855129 22871 heartbeater.cc:347] Connected to a master server at 127.20.115.254:39003
I20260924 21:16:54.855227 22871 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:54.855476 22871 heartbeater.cc:511] Master 127.20.115.254:39003 requested a full tablet report, sending...
I20260924 21:16:54.856163 22667 ts_manager.cc:194] Registered new tserver with Master: 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673)
I20260924 21:16:54.857174 22667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:47711
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
I20260924 21:16:54.862792 22878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:54.856223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39003 (local address 127.20.115.193:36035)
0924 21:16:54.856907 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:54.856918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:54.856985 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:54.857117 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:54.857117 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:54.857297 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:54.857346 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:54.857347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.858375 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.858376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:54.858861 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:54.858863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.859341 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.859342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:54.859399 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:54.859775 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:54.859779 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:54.860047 (+ 268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:54.861744 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:54.861745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:54.861745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:54.861985 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:54.862223 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:54.862223 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:54.862223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:54.862279 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:54.862560 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:54.862561 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:54.862597 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 21:16:54.862648 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:16:54.862724 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":71,"threads_started":1}
I20260924 21:16:54.862888 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:54.856271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:36035 (local address 127.20.115.254:39003)
0924 21:16:54.856309 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:54.856311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:54.856971 (+ 660us) server_negotiation.cc:407] Connection header received
0924 21:16:54.857042 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:54.857043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:54.857053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:54.857111 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:54.857478 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.858256 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.858945 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.859264 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.859344 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:54.860191 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:54.860192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:54.860192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:54.860198 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:54.861666 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:54.862083 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:54.862084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:54.862085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:54.862152 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:54.862339 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:54.862340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:54.862340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:54.862495 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:54.862566 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:54.862673 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 21:16:54.862753 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:16:54.878062 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:39003
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:54.959553 22880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:54.959758 22880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:54.959782 22880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:54.960969 22880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:54.961014 22880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:54.961033 22880 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 21:16:54.961045 22880 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:16:54.961063 22880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:16:54.962581 22880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.115.254:39003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:54.962818 22880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:54.963056 22880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:54.968103 22888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:54.968115 22885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:54.968115 22886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:54.968204 22880 server_base.cc:1034] running on GCE node
I20260924 21:16:54.968631 22880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:54.968909 22880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:54.970077 22880 hybrid_clock.cc:648] HybridClock initialized: now 1790284614970069 us; error 41 us; skew 500 ppm
Sep 24 21:16:54 dist-test-slave-cxpd krb5kdc[22623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284614, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:54.972625 22880 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:16:54.973196 22880 webserver.cc:460] Webserver started at http://127.20.115.194:36857/ using document root <none> and password file <none>
I20260924 21:16:54.973441 22880 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:54.973510 22880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:54.973665 22880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:54.974717 22880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "6030eabf018a4316b6925487fac5b422"
format_stamp: "Formatted at 2026-09-24 21:16:54 on dist-test-slave-cxpd"
server_key: "c112bbae679e84861e919f775dffe241"
server_key_iv: "128dae2f1bc69579550cc29a8d3e22f4"
server_key_version: "encryptionkey@0"
I20260924 21:16:54.975107 22880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "6030eabf018a4316b6925487fac5b422"
format_stamp: "Formatted at 2026-09-24 21:16:54 on dist-test-slave-cxpd"
server_key: "c112bbae679e84861e919f775dffe241"
server_key_iv: "128dae2f1bc69579550cc29a8d3e22f4"
server_key_version: "encryptionkey@0"
I20260924 21:16:54.976449 22880 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:16:54.977332 22895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:54.977525 22880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:16:54.977631 22880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "6030eabf018a4316b6925487fac5b422"
format_stamp: "Formatted at 2026-09-24 21:16:54 on dist-test-slave-cxpd"
server_key: "c112bbae679e84861e919f775dffe241"
server_key_iv: "128dae2f1bc69579550cc29a8d3e22f4"
server_key_version: "encryptionkey@0"
I20260924 21:16:54.977716 22880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:54.989244 22880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:54.991086 22880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:54.991237 22880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:54.991827 22880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:54.991878 22880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:54.991923 22880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:54.991948 22880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:16:54 dist-test-slave-cxpd krb5kdc[22623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284614, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:54.999557 22880 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:40305
I20260924 21:16:54.999688 23010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:40305 every 8 connection(s)
I20260924 21:16:55.000113 22880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:16:55.002311 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:54.992347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:54511 (local address 127.20.115.254:39003)
0924 21:16:54.992432 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:54.992434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:54.992806 (+ 372us) server_negotiation.cc:407] Connection header received
0924 21:16:54.993445 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:54.993447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:54.993460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:54.993528 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:54.994425 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.994945 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.996598 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.996886 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.996968 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:54.999237 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:54.999238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:54.999239 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:54.999247 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:55.000918 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.001334 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.001336 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.001336 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.001393 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.001657 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.001658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.001658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.001779 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:55.001871 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.002139 (+ 268us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.002217 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:16:55.002518 22904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:54.992331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39003 (local address 127.20.115.194:54511)
0924 21:16:54.992543 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:54.992592 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:54.993319 (+ 727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:54.993611 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:54.993617 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:54.994012 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:54.994314 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:54.994322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.995039 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.995040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:54.996500 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:54.996503 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:54.997002 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:54.997003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:54.997142 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:54.997758 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:54.997765 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:54.999135 (+ 1370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:55.001009 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.001016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.001018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.001241 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.001467 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.001468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.001468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.001587 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.001864 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:55.001870 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:55.002009 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.002119 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.002277 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":105,"threads_started":1}
I20260924 21:16:55.003038 23011 heartbeater.cc:347] Connected to a master server at 127.20.115.254:39003
I20260924 21:16:55.003090 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 22880
I20260924 21:16:55.003158 23011 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:55.003191 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:16:55.003345 23011 heartbeater.cc:511] Master 127.20.115.254:39003 requested a full tablet report, sending...
I20260924 21:16:55.003382 20943 external_mini_cluster.cc:1428] Setting key eb3891844db4aeac34bbb55d77d5c86b
I20260924 21:16:55.003801 22667 ts_manager.cc:194] Registered new tserver with Master: 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305)
I20260924 21:16:55.004357 22667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:54511
I20260924 21:16:55.010589 23017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.004064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39003 (local address 127.20.115.194:41883)
0924 21:16:55.004188 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.004197 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.004248 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.004360 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.004361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.004558 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:55.004622 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.004623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.005802 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.005803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.006461 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.006464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.006924 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.006925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.006990 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.007523 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:55.007528 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:55.007809 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:55.009519 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.009520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.009520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.009690 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.009895 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.009896 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.009896 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.009954 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.010244 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:55.010246 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:55.010293 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.010383 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.010491 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260924 21:16:55.010583 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.004105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:41883 (local address 127.20.115.254:39003)
0924 21:16:55.004138 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.004140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.004259 (+ 119us) server_negotiation.cc:407] Connection header received
0924 21:16:55.004288 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.004289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.004299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.004358 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:55.005227 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.005703 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.006541 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.006827 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.006899 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.007900 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:55.007902 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:55.007902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:55.007916 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:55.009371 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.009772 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.009773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.009773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.009828 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.010027 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.010028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.010028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.010172 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:55.010253 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.010410 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.010494 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:16:55.032618 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:39003
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:16:55.114295 23020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:16:55.114506 23020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:16:55.114531 23020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:16:55.115718 23020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:16:55.115762 23020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:16:55.115783 23020 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260924 21:16:55.115821 23020 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:16:55.115864 23020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:16:55.117375 23020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.115.254:39003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:16:55.117632 23020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:16:55.117872 23020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:16:55.123010 23028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:55.123062 23026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:16:55.123085 23025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:16:55.123145 23020 server_base.cc:1034] running on GCE node
I20260924 21:16:55.123404 23020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:16:55.123624 23020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:16:55.124744 23020 hybrid_clock.cc:648] HybridClock initialized: now 1790284615124728 us; error 46 us; skew 500 ppm
Sep 24 21:16:55 dist-test-slave-cxpd krb5kdc[22623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284615, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:16:55.127332 23020 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:16:55.127810 23020 webserver.cc:460] Webserver started at http://127.20.115.195:36833/ using document root <none> and password file <none>
I20260924 21:16:55.128038 23020 fs_manager.cc:359] Metadata directory not provided
I20260924 21:16:55.128083 23020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:16:55.128227 23020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:16:55.129364 23020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "ad224586aa2343999b9a72f6cb305c5a"
format_stamp: "Formatted at 2026-09-24 21:16:55 on dist-test-slave-cxpd"
server_key: "e67a9e059b263bf3b87a6a83a2584daa"
server_key_iv: "ea9b9372f50b4986e4bee03b2a5717c3"
server_key_version: "encryptionkey@0"
I20260924 21:16:55.129730 23020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "ad224586aa2343999b9a72f6cb305c5a"
format_stamp: "Formatted at 2026-09-24 21:16:55 on dist-test-slave-cxpd"
server_key: "e67a9e059b263bf3b87a6a83a2584daa"
server_key_iv: "ea9b9372f50b4986e4bee03b2a5717c3"
server_key_version: "encryptionkey@0"
I20260924 21:16:55.131042 23020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:16:55.131815 23035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:55.132056 23020 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:55.132135 23020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "ad224586aa2343999b9a72f6cb305c5a"
format_stamp: "Formatted at 2026-09-24 21:16:55 on dist-test-slave-cxpd"
server_key: "e67a9e059b263bf3b87a6a83a2584daa"
server_key_iv: "ea9b9372f50b4986e4bee03b2a5717c3"
server_key_version: "encryptionkey@0"
I20260924 21:16:55.132189 23020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:16:55.148495 23020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:16:55.150586 23020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:16:55.150702 23020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:16:55.151172 23020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:16:55.151213 23020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:55.151237 23020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:16:55.151250 23020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:55.158305 23020 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:39861
I20260924 21:16:55.158380 23150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:39861 every 8 connection(s)
I20260924 21:16:55.158715 23020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:16:55 dist-test-slave-cxpd krb5kdc[22623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284615, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:55.162072 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.151743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:49341 (local address 127.20.115.254:39003)
0924 21:16:55.151817 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.151819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.152067 (+ 248us) server_negotiation.cc:407] Connection header received
0924 21:16:55.152660 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.152662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.152673 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.152755 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:55.153780 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.154643 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.156387 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.156649 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.156717 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.159432 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:55.159433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:55.159434 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:55.159442 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:55.160935 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.161324 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.161325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.161325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.161383 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.161571 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.161571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.161572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.161668 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:55.161716 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.161922 (+ 206us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.162009 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 21:16:55.162201 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.151734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39003 (local address 127.20.115.195:49341)
0924 21:16:55.151952 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.151992 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.152546 (+ 554us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.152940 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.152946 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.153353 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:55.153680 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.153686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.154750 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.154751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.156290 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.156293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.156753 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.156754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.156966 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.157550 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:55.157561 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:55.159297 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:55.161020 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.161025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.161027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.161242 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.161443 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.161443 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.161444 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.161533 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.161704 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:55.161707 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:55.161814 (+ 107us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.161892 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.162017 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":153,"threads_started":1}
I20260924 21:16:55.162732 23151 heartbeater.cc:347] Connected to a master server at 127.20.115.254:39003
I20260924 21:16:55.162829 23151 heartbeater.cc:464] Registering TS with master...
I20260924 21:16:55.163002 23151 heartbeater.cc:511] Master 127.20.115.254:39003 requested a full tablet report, sending...
I20260924 21:16:55.163388 22667 ts_manager.cc:194] Registered new tserver with Master: ad224586aa2343999b9a72f6cb305c5a (127.20.115.195:39861)
I20260924 21:16:55.163766 22667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:49341
I20260924 21:16:55.167678 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 23020
I20260924 21:16:55.167783 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:16:55.167949 20943 external_mini_cluster.cc:1428] Setting key cc50b42fb10c11d9925040a988726780
I20260924 21:16:55.168962 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:16:55.169543 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.163870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:51867 (local address 127.20.115.254:39003)
0924 21:16:55.163908 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.163910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.164022 (+ 112us) server_negotiation.cc:407] Connection header received
0924 21:16:55.164044 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.164045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.164056 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.164117 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:55.164488 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.165060 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.165732 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.165908 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.165961 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.166844 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:55.166845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:55.166845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:55.166849 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:55.168276 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.168688 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.168690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.168690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.168752 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.169001 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.169003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.169003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.169142 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:55.169221 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.169377 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.169454 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:16:55.169560 23157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.163822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39003 (local address 127.20.115.195:51867)
0924 21:16:55.163940 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.163950 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.164009 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.164151 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.164152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.164345 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:55.164402 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.164404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.165150 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.165151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.165660 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.165661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.165977 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.165978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.166034 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.166493 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:55.166498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:55.166778 (+ 280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:55.168404 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.168405 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.168405 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.168618 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.168859 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.168859 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.168860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.168925 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.169212 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:55.169213 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:55.169259 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.169350 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.169468 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
Sep 24 21:16:55 dist-test-slave-cxpd krb5kdc[22623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:16:55 dist-test-slave-cxpd krb5kdc[22623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:55.192055 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.185283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.20.115.254:39003)
0924 21:16:55.185354 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.185357 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.185367 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:55.185444 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.185445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.185463 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.185524 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:55.186147 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.186649 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.187404 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.187597 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.187676 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.189982 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:55.189983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:55.189984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:55.189990 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:55.191145 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.191461 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.191461 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.191462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.191508 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.191672 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.191673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.191673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.191768 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:55.191850 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.191933 (+ 83us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.191990 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:16:55.193699 22667 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:55.195197 22667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:55.201651 23170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.197452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40305 (local address 127.0.0.1:41918)
0924 21:16:55.197883 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.197895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.197953 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.198265 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.198269 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.198271 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:55.198348 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.198351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.199263 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.199264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.200423 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.200425 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.201339 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.201340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.201419 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.201433 (+ 14us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.201523 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.201570 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":72,"threads_started":1}
I20260924 21:16:55.201694 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.197706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41918 (local address 127.20.115.194:40305)
0924 21:16:55.197969 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.197980 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.197990 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:55.198077 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.198079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.198096 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.198176 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:55.198414 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.199161 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.200509 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.201251 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.201330 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.201436 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.201542 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.201590 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":86,"threads_started":1}
I20260924 21:16:55.201850 23168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.197427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:39861 (local address 127.0.0.1:45572)
0924 21:16:55.197699 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.197747 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.197823 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.198370 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.198371 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.198373 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:55.198416 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.198418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.199744 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.199745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.200717 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.200719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.201628 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.201629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.201699 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.201704 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.201735 (+ 31us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.201783 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":79,"threads_started":1}
I20260924 21:16:55.202184 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.197891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45572 (local address 127.20.115.195:39861)
0924 21:16:55.198206 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.198210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.198220 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:16:55.198258 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.198260 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.198274 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.198479 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:55.198548 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.199658 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.200817 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.201533 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.201991 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.202057 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.202087 (+ 30us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.202117 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":126,"threads_started":1}
I20260924 21:16:55.202729 22945 tablet_service.cc:1467] Processing CreateTablet for tablet 2d36b7cd7ebd407b9ee9e15d40ed2693 (DEFAULT_TABLE table=test-table [id=b3581b7071ec4887971acb7466b4e837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:55.202894 23085 tablet_service.cc:1467] Processing CreateTablet for tablet 2d36b7cd7ebd407b9ee9e15d40ed2693 (DEFAULT_TABLE table=test-table [id=b3581b7071ec4887971acb7466b4e837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:55.203109 22945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d36b7cd7ebd407b9ee9e15d40ed2693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:55.203238 23085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d36b7cd7ebd407b9ee9e15d40ed2693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:55.203238 23169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.197548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:43673 (local address 127.0.0.1:46828)
0924 21:16:55.198032 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.198041 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.198082 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.198378 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.198379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.198381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:55.198428 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.198430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.200831 (+ 2401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.200832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.201652 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.201654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.203008 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.203010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.203072 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.203076 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.203126 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.203165 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":91,"threads_started":1}
I20260924 21:16:55.203464 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.197718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46828 (local address 127.20.115.193:43673)
0924 21:16:55.197894 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.197900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.198167 (+ 267us) server_negotiation.cc:407] Connection header received
0924 21:16:55.198216 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.198218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.198233 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.198306 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:55.198513 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.199650 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.201874 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.202755 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.202805 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.202850 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.203338 (+ 488us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.203379 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260924 21:16:55.206070 23174 tablet_bootstrap.cc:492] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: Bootstrap starting.
I20260924 21:16:55.206789 22805 tablet_service.cc:1467] Processing CreateTablet for tablet 2d36b7cd7ebd407b9ee9e15d40ed2693 (DEFAULT_TABLE table=test-table [id=b3581b7071ec4887971acb7466b4e837]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:55.207105 23174 tablet_bootstrap.cc:654] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:55.207105 22805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2d36b7cd7ebd407b9ee9e15d40ed2693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:55.207434 23174 log.cc:824] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:55.208190 23174 tablet_bootstrap.cc:492] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: No bootstrap required, opened a new log
I20260924 21:16:55.208261 23174 ts_tablet_manager.cc:1397] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:16:55.209051 23175 tablet_bootstrap.cc:492] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: Bootstrap starting.
I20260924 21:16:55.209589 23175 tablet_bootstrap.cc:654] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:55.209893 23175 log.cc:824] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:55.210361 23174 raft_consensus.cc:348] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } }
I20260924 21:16:55.210527 23174 raft_consensus.cc:374] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:55.210579 23174 raft_consensus.cc:729] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad224586aa2343999b9a72f6cb305c5a, State: Initialized, Role: FOLLOWER
I20260924 21:16:55.210606 23175 tablet_bootstrap.cc:492] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: No bootstrap required, opened a new log
I20260924 21:16:55.210672 23175 ts_tablet_manager.cc:1397] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:55.210712 23174 consensus_queue.cc:260] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } }
I20260924 21:16:55.210930 23178 tablet_bootstrap.cc:492] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: Bootstrap starting.
I20260924 21:16:55.210992 23174 ts_tablet_manager.cc:1428] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:55.211054 23151 heartbeater.cc:503] Master 127.20.115.254:39003 was elected leader, sending a full tablet report...
I20260924 21:16:55.211510 23178 tablet_bootstrap.cc:654] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:55.211812 23178 log.cc:824] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: Log is configured to *not* fsync() on all Append() calls
I20260924 21:16:55.212528 23178 tablet_bootstrap.cc:492] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: No bootstrap required, opened a new log
I20260924 21:16:55.212577 23178 ts_tablet_manager.cc:1397] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:55.212893 23175 raft_consensus.cc:348] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } }
I20260924 21:16:55.213049 23175 raft_consensus.cc:374] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:55.213093 23175 raft_consensus.cc:729] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6030eabf018a4316b6925487fac5b422, State: Initialized, Role: FOLLOWER
I20260924 21:16:55.213201 23175 consensus_queue.cc:260] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } }
I20260924 21:16:55.213470 23175 ts_tablet_manager.cc:1428] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:16:55.213481 23011 heartbeater.cc:503] Master 127.20.115.254:39003 was elected leader, sending a full tablet report...
I20260924 21:16:55.214504 23178 raft_consensus.cc:348] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } }
I20260924 21:16:55.214612 23178 raft_consensus.cc:374] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:55.214632 23178 raft_consensus.cc:729] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c42c83fd7f4af29c1bdf3db5b81403, State: Initialized, Role: FOLLOWER
I20260924 21:16:55.214700 23178 consensus_queue.cc:260] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } }
I20260924 21:16:55.214892 23178 ts_tablet_manager.cc:1428] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:16:55.214969 22871 heartbeater.cc:503] Master 127.20.115.254:39003 was elected leader, sending a full tablet report...
W20260924 21:16:55.250721 23012 tablet.cc:2369] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:55.296981 23179 raft_consensus.cc:482] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:55.297117 23179 raft_consensus.cc:504] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } }
I20260924 21:16:55.297571 23179 leader_election.cc:288] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673), 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305)
I20260924 21:16:55.301870 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.298041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:41783 (local address 127.20.115.193:43673)
0924 21:16:55.298130 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.298133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.298359 (+ 226us) server_negotiation.cc:407] Connection header received
0924 21:16:55.298480 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.298482 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.298493 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.298526 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:55.298759 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.299517 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.300838 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.301463 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.301551 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.301560 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.301744 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.301786 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 21:16:55.301862 23171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.298053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:58829 (local address 127.20.115.194:40305)
0924 21:16:55.298141 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.298143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.298154 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:16:55.298212 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.298214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.298226 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.298312 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:55.298497 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.299370 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.300857 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.301486 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.301553 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.301560 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.301744 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.301786 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 21:16:55.301868 23044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.297884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40305 (local address 127.20.115.195:58829)
0924 21:16:55.297931 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.297977 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.298031 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.298307 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.298308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.298311 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:55.298394 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.298396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.299476 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.299477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.300731 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.300734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.301569 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.301571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.301662 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.301665 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.301722 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.301769 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":12}
I20260924 21:16:55.301877 23183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.297885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:43673 (local address 127.20.115.195:41783)
0924 21:16:55.298288 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.298299 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.298391 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.298600 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.298600 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.298602 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:55.298683 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.298684 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.299610 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.299610 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.300731 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.300734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.301570 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.301571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.301652 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.301656 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.301722 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.301769 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":213,"thread_start_us":133,"threads_started":1}
I20260924 21:16:55.302304 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d36b7cd7ebd407b9ee9e15d40ed2693" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422" is_pre_election: true
I20260924 21:16:55.302452 22965 raft_consensus.cc:2454] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 0.
I20260924 21:16:55.302632 22825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d36b7cd7ebd407b9ee9e15d40ed2693" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" is_pre_election: true
I20260924 21:16:55.302670 23039 leader_election.cc:302] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6030eabf018a4316b6925487fac5b422, ad224586aa2343999b9a72f6cb305c5a; no voters:
I20260924 21:16:55.302804 22825 raft_consensus.cc:2454] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 0.
I20260924 21:16:55.302875 23179 raft_consensus.cc:2781] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:55.302954 23179 raft_consensus.cc:482] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:55.302989 23179 raft_consensus.cc:3037] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:55.303743 23179 raft_consensus.cc:504] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } }
I20260924 21:16:55.303880 23179 leader_election.cc:288] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 election: Requested vote from peers 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673), 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305)
I20260924 21:16:55.304006 22825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d36b7cd7ebd407b9ee9e15d40ed2693" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50c42c83fd7f4af29c1bdf3db5b81403"
I20260924 21:16:55.304041 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2d36b7cd7ebd407b9ee9e15d40ed2693" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422"
I20260924 21:16:55.304095 22825 raft_consensus.cc:3037] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:55.304126 22965 raft_consensus.cc:3037] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:55.304867 22965 raft_consensus.cc:2454] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 1.
I20260924 21:16:55.304898 22825 raft_consensus.cc:2454] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 1.
I20260924 21:16:55.305075 23039 leader_election.cc:302] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6030eabf018a4316b6925487fac5b422, ad224586aa2343999b9a72f6cb305c5a; no voters:
I20260924 21:16:55.305172 23179 raft_consensus.cc:2781] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:55.305308 23179 raft_consensus.cc:686] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 1 LEADER]: Becoming Leader. State: Replica: ad224586aa2343999b9a72f6cb305c5a, State: Running, Role: LEADER
I20260924 21:16:55.305433 23179 consensus_queue.cc:237] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } }
I20260924 21:16:55.306399 22668 catalog_manager.cc:5495] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a reported cstate change: term changed from 0 to 1, leader changed from <none> to ad224586aa2343999b9a72f6cb305c5a (127.20.115.195). New cstate: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } health_report { overall_health: UNKNOWN } } }
Sep 24 21:16:55 dist-test-slave-cxpd krb5kdc[22623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284614, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:55.344287 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.337441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39003 (local address 127.0.0.1:54740)
0924 21:16:55.337645 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.337655 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.337721 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.337858 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.337859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.338157 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:16:55.338202 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.338205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.338831 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.338832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.339365 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.339367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.339693 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.339695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.339756 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.340232 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:16:55.340237 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:16:55.341609 (+ 1372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:16:55.343053 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.343061 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.343063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.343254 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.343553 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:16:55.343554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:16:55.343555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:16:55.343618 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:16:55.343841 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:16:55.343846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:16:55.343895 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.343976 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.344129 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":2,"thread_start_us":106,"threads_started":1}
I20260924 21:16:55.344295 22717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.337506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54740 (local address 127.20.115.254:39003)
0924 21:16:55.337572 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.337575 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.337717 (+ 142us) server_negotiation.cc:407] Connection header received
0924 21:16:55.337782 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.337783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.337791 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.337841 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:55.338266 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.338745 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.339438 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.339615 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.339665 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.341699 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:55.341701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:55.341701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:55.341708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:55.342951 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.343341 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.343342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.343343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.343396 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:55.343681 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:55.343681 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:55.343682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:55.343779 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:55.343830 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.343989 (+ 159us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.344186 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":40}
I20260924 21:16:55.346406 22668 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:54740:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 21:16:55.346599 22668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:55.348037 22805 tablet_service.cc:1467] Processing CreateTablet for tablet 318a402995574f28bc98cad7423bc695 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c6b46984cdf74a9c861bd6af5b8f5525]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:16:55.348076 22945 tablet_service.cc:1467] Processing CreateTablet for tablet 318a402995574f28bc98cad7423bc695 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c6b46984cdf74a9c861bd6af5b8f5525]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:16:55.348220 22945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 318a402995574f28bc98cad7423bc695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:55.348167 23085 tablet_service.cc:1467] Processing CreateTablet for tablet 318a402995574f28bc98cad7423bc695 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c6b46984cdf74a9c861bd6af5b8f5525]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:16:55.348225 22805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 318a402995574f28bc98cad7423bc695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:55.348299 23085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 318a402995574f28bc98cad7423bc695. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:55.349529 23175 tablet_bootstrap.cc:492] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: Bootstrap starting.
I20260924 21:16:55.349821 23178 tablet_bootstrap.cc:492] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: Bootstrap starting.
I20260924 21:16:55.350015 23174 tablet_bootstrap.cc:492] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: Bootstrap starting.
I20260924 21:16:55.350279 23175 tablet_bootstrap.cc:654] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:55.350442 23174 tablet_bootstrap.cc:654] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:55.350441 23178 tablet_bootstrap.cc:654] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: Neither blocks nor log segments found. Creating new log.
W20260924 21:16:55.350741 22872 tablet.cc:2369] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:55.351081 23174 tablet_bootstrap.cc:492] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: No bootstrap required, opened a new log
I20260924 21:16:55.351100 23178 tablet_bootstrap.cc:492] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: No bootstrap required, opened a new log
I20260924 21:16:55.351147 23178 ts_tablet_manager.cc:1397] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:16:55.351148 23174 ts_tablet_manager.cc:1397] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:55.351313 23175 tablet_bootstrap.cc:492] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: No bootstrap required, opened a new log
I20260924 21:16:55.351303 23178 raft_consensus.cc:348] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.351297 23174 raft_consensus.cc:348] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.351361 23175 ts_tablet_manager.cc:1397] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:55.351380 23178 raft_consensus.cc:374] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:55.351385 23174 raft_consensus.cc:374] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:55.351415 23178 raft_consensus.cc:729] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c42c83fd7f4af29c1bdf3db5b81403, State: Initialized, Role: FOLLOWER
I20260924 21:16:55.351415 23174 raft_consensus.cc:729] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad224586aa2343999b9a72f6cb305c5a, State: Initialized, Role: FOLLOWER
I20260924 21:16:55.351470 23174 consensus_queue.cc:260] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.351480 23178 consensus_queue.cc:260] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.351500 23175 raft_consensus.cc:348] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.351557 23174 ts_tablet_manager.cc:1428] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:55.351565 23178 ts_tablet_manager.cc:1428] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:55.351567 23175 raft_consensus.cc:374] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:55.351593 23175 raft_consensus.cc:729] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6030eabf018a4316b6925487fac5b422, State: Initialized, Role: FOLLOWER
I20260924 21:16:55.351643 23175 consensus_queue.cc:260] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.351689 23179 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.351748 23179 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:55.351764 23175 ts_tablet_manager.cc:1428] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:55.352043 23182 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.352131 23182 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:55.352274 23181 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.352375 23181 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: This TxnStatusTablet replica's current role is: FOLLOWER
W20260924 21:16:55.409361 23152 tablet.cc:2369] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:16:55.552982 23179 raft_consensus.cc:482] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:55.553081 23179 raft_consensus.cc:504] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.553246 23179 leader_election.cc:288] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673), 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305)
I20260924 21:16:55.553478 22825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318a402995574f28bc98cad7423bc695" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" is_pre_election: true
I20260924 21:16:55.553478 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318a402995574f28bc98cad7423bc695" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422" is_pre_election: true
I20260924 21:16:55.553581 22965 raft_consensus.cc:2454] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 0.
I20260924 21:16:55.553583 22825 raft_consensus.cc:2454] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 0.
I20260924 21:16:55.553732 23039 leader_election.cc:302] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6030eabf018a4316b6925487fac5b422, ad224586aa2343999b9a72f6cb305c5a; no voters:
I20260924 21:16:55.553887 23179 raft_consensus.cc:2781] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:55.553930 23179 raft_consensus.cc:482] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:55.553954 23179 raft_consensus.cc:3037] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:55.554642 23179 raft_consensus.cc:504] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.554785 23179 leader_election.cc:288] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 election: Requested vote from peers 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673), 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305)
I20260924 21:16:55.554949 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318a402995574f28bc98cad7423bc695" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422"
I20260924 21:16:55.554975 22825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "318a402995574f28bc98cad7423bc695" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50c42c83fd7f4af29c1bdf3db5b81403"
I20260924 21:16:55.555029 22965 raft_consensus.cc:3037] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:55.555049 22825 raft_consensus.cc:3037] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:55.555703 22825 raft_consensus.cc:2454] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 1.
I20260924 21:16:55.555743 22965 raft_consensus.cc:2454] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 1.
I20260924 21:16:55.555932 23039 leader_election.cc:302] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6030eabf018a4316b6925487fac5b422, ad224586aa2343999b9a72f6cb305c5a; no voters:
I20260924 21:16:55.556052 23179 raft_consensus.cc:2781] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:55.556115 23179 raft_consensus.cc:686] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [term 1 LEADER]: Becoming Leader. State: Replica: ad224586aa2343999b9a72f6cb305c5a, State: Running, Role: LEADER
I20260924 21:16:55.556170 23179 consensus_queue.cc:237] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.556490 23184 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: TxnStatusTablet state changed. Reason: New leader ad224586aa2343999b9a72f6cb305c5a. Latest consensus state: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.556581 23184 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:16:55.556807 23196 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 21:16:55.556881 22667 catalog_manager.cc:5495] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a reported cstate change: term changed from 0 to 1, leader changed from <none> to ad224586aa2343999b9a72f6cb305c5a (127.20.115.195). New cstate: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260924 21:16:55.556972 23196 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 21:16:55.580348 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.577829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45580 (local address 127.20.115.195:39861)
0924 21:16:55.577911 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.577913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.577922 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:16:55.577957 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.577958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.577979 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.578044 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:55.578207 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.578676 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.579504 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.579718 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.579795 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.579978 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:55.580096 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:55.580180 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.580234 (+ 54us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.580256 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 21:16:55.580348 23191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.577719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:39861 (local address 127.0.0.1:45580)
0924 21:16:55.577816 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.577828 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.577883 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.578072 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.578073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.578074 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:16:55.578124 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.578127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.578761 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.578762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.579418 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.579420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.579796 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.579797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.579863 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.579878 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:16:55.580177 (+ 299us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:16:55.580186 (+ 9us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.580232 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.580276 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260924 21:16:55.582494 22965 raft_consensus.cc:1261] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Refusing update from remote peer ad224586aa2343999b9a72f6cb305c5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:55.582515 22825 raft_consensus.cc:1261] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Refusing update from remote peer ad224586aa2343999b9a72f6cb305c5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:55.582839 23179 consensus_queue.cc:1035] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:55.582988 23184 consensus_queue.cc:1035] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:55.584091 23182 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: TxnStatusTablet state changed. Reason: New leader ad224586aa2343999b9a72f6cb305c5a. Latest consensus state: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.584111 23181 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: TxnStatusTablet state changed. Reason: New leader ad224586aa2343999b9a72f6cb305c5a. Latest consensus state: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.584178 23182 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:55.584194 23181 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:55.585136 23184 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.585223 23184 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:16:55.585294 23199 mvcc.cc:205] Tried to move back new op lower bound from 7333005785423028224 to 7333005785318346752. Current Snapshot: MvccSnapshot[applied={T|T < 7333005785423028224}]
I20260924 21:16:55.585703 23182 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.585776 23182 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P 50c42c83fd7f4af29c1bdf3db5b81403: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:55.586138 23184 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.586215 23184 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P ad224586aa2343999b9a72f6cb305c5a: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:16:55.586963 23181 tablet_replica.cc:439] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } } }
I20260924 21:16:55.587044 23181 tablet_replica.cc:442] T 318a402995574f28bc98cad7423bc695 P 6030eabf018a4316b6925487fac5b422: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:16:55.590206 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.587747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45584 (local address 127.20.115.195:39861)
0924 21:16:55.587791 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.587793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.587801 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:16:55.587858 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.587859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.587866 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.587916 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:55.588037 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.588465 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.589593 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.589779 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.589835 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.590001 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:55.590025 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:55.590062 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.590115 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.590130 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260924 21:16:55.595067 23157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.592531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:39861 (local address 127.20.115.195:58481)
0924 21:16:55.592608 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:55.592620 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:55.592668 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:55.592795 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:55.592795 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:55.592796 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:16:55.592854 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.592855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.593438 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.593439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.594272 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:55.594274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.594621 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.594622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:55.594680 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.594694 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:16:55.594919 (+ 225us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:16:55.594921 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:16:55.594962 (+ 41us) client_negotiation.cc:240] Negotiation successful
0924 21:16:55.595002 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 21:16:55.595067 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:55.592592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:58481 (local address 127.20.115.195:39861)
0924 21:16:55.592631 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:55.592633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:55.592655 (+ 22us) server_negotiation.cc:407] Connection header received
0924 21:16:55.592721 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:55.592722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:55.592729 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:55.592766 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:55.592928 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.593356 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.594365 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:55.594553 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:55.594624 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:55.594789 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:55.594832 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:55.594912 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:55.594973 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 21:16:55.594993 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":24}
I20260924 21:16:55.597258 22965 raft_consensus.cc:1261] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Refusing update from remote peer ad224586aa2343999b9a72f6cb305c5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:55.597288 22825 raft_consensus.cc:1261] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Refusing update from remote peer ad224586aa2343999b9a72f6cb305c5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:55.597446 23184 consensus_queue.cc:1035] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:55.597554 23179 consensus_queue.cc:1035] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:55.598964 23200 mvcc.cc:205] Tried to move back new op lower bound from 7333005785485123584 to 7333005784291778560. Current Snapshot: MvccSnapshot[applied={T|T < 7333005785485123584}]
I20260924 21:16:55.605656 23201 mvcc.cc:205] Tried to move back new op lower bound from 7333005785485123584 to 7333005784291778560. Current Snapshot: MvccSnapshot[applied={T|T < 7333005785485123584}]
I20260924 21:16:55.610723 22667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:55.610819 22667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:55.611505 22667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47: Sending DeleteTablet for 3 replicas of tablet 2d36b7cd7ebd407b9ee9e15d40ed2693
I20260924 21:16:55.611799 22805 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d36b7cd7ebd407b9ee9e15d40ed2693 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:55 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:46828
I20260924 21:16:55.611815 23085 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d36b7cd7ebd407b9ee9e15d40ed2693 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:55 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:45572
I20260924 21:16:55.612056 23215 tablet_replica.cc:330] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: stopping tablet replica
I20260924 21:16:55.612156 23215 raft_consensus.cc:2229] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:55.612259 23215 raft_consensus.cc:2258] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:55.612250 22945 tablet_service.cc:1514] Processing DeleteTablet for tablet 2d36b7cd7ebd407b9ee9e15d40ed2693 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:55 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:41918
I20260924 21:16:55.612250 22667 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:55.612393 22667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:55.612512 23216 tablet_replica.cc:330] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: stopping tablet replica
I20260924 21:16:55.612928 23215 ts_tablet_manager.cc:1905] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:55.613746 23085 tablet_service.cc:1467] Processing CreateTablet for tablet dd210bbb6e474c2d873ce26c1b1f692a (DEFAULT_TABLE table=test-table [id=a7be71e76c924f42b6c83fddcf24ef6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:55.613765 22945 tablet_service.cc:1467] Processing CreateTablet for tablet dd210bbb6e474c2d873ce26c1b1f692a (DEFAULT_TABLE table=test-table [id=a7be71e76c924f42b6c83fddcf24ef6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:55.613862 22945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd210bbb6e474c2d873ce26c1b1f692a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:55.614017 23085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd210bbb6e474c2d873ce26c1b1f692a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:55.615031 23215 ts_tablet_manager.cc:1918] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:16:55.615096 23215 log.cc:1196] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/2d36b7cd7ebd407b9ee9e15d40ed2693
I20260924 21:16:55.615218 23175 tablet_bootstrap.cc:492] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: Bootstrap starting.
I20260924 21:16:55.615239 23174 tablet_bootstrap.cc:492] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: Bootstrap starting.
I20260924 21:16:55.615346 23215 ts_tablet_manager.cc:1939] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P ad224586aa2343999b9a72f6cb305c5a: Deleting consensus metadata
I20260924 21:16:55.615628 23174 tablet_bootstrap.cc:654] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:55.615628 23175 tablet_bootstrap.cc:654] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:55.615821 22805 tablet_service.cc:1467] Processing CreateTablet for tablet dd210bbb6e474c2d873ce26c1b1f692a (DEFAULT_TABLE table=test-table [id=a7be71e76c924f42b6c83fddcf24ef6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:55.615926 22805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd210bbb6e474c2d873ce26c1b1f692a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:55.616073 22653 catalog_manager.cc:4841] TS ad224586aa2343999b9a72f6cb305c5a (127.20.115.195:39861): tablet 2d36b7cd7ebd407b9ee9e15d40ed2693 (table test-table [id=b3581b7071ec4887971acb7466b4e837]) successfully deleted
I20260924 21:16:55.616636 23175 tablet_bootstrap.cc:492] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: No bootstrap required, opened a new log
I20260924 21:16:55.616683 23175 ts_tablet_manager.cc:1397] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:55.616791 23174 tablet_bootstrap.cc:492] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: No bootstrap required, opened a new log
I20260924 21:16:55.616825 23174 ts_tablet_manager.cc:1397] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:16:55.616861 23175 raft_consensus.cc:348] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.616926 23175 raft_consensus.cc:374] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:55.616946 23175 raft_consensus.cc:729] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6030eabf018a4316b6925487fac5b422, State: Initialized, Role: FOLLOWER
I20260924 21:16:55.616956 23174 raft_consensus.cc:348] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.617024 23174 raft_consensus.cc:374] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:55.617002 23175 consensus_queue.cc:260] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.617046 23174 raft_consensus.cc:729] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad224586aa2343999b9a72f6cb305c5a, State: Initialized, Role: FOLLOWER
I20260924 21:16:55.617089 23175 ts_tablet_manager.cc:1428] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:55.617087 23174 consensus_queue.cc:260] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.617169 23174 ts_tablet_manager.cc:1428] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:55.617237 23178 tablet_bootstrap.cc:492] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: Bootstrap starting.
I20260924 21:16:55.617609 23178 tablet_bootstrap.cc:654] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:55.618081 23216 raft_consensus.cc:2229] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:55.618196 23216 raft_consensus.cc:2258] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:55.618592 23216 ts_tablet_manager.cc:1905] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:55.620018 23216 ts_tablet_manager.cc:1918] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:16:55.620090 23216 log.cc:1196] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/2d36b7cd7ebd407b9ee9e15d40ed2693
I20260924 21:16:55.620327 23216 ts_tablet_manager.cc:1939] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 6030eabf018a4316b6925487fac5b422: Deleting consensus metadata
I20260924 21:16:55.620728 22655 catalog_manager.cc:4841] TS 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305): tablet 2d36b7cd7ebd407b9ee9e15d40ed2693 (table test-table [id=b3581b7071ec4887971acb7466b4e837]) successfully deleted
I20260924 21:16:55.620898 23214 tablet_replica.cc:330] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: stopping tablet replica
I20260924 21:16:55.621033 23214 raft_consensus.cc:2229] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:55.621127 23214 raft_consensus.cc:2258] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:55.621196 23178 tablet_bootstrap.cc:492] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: No bootstrap required, opened a new log
I20260924 21:16:55.621244 23178 ts_tablet_manager.cc:1397] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260924 21:16:55.621378 23178 raft_consensus.cc:348] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.621450 23178 raft_consensus.cc:374] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:55.621470 23178 raft_consensus.cc:729] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c42c83fd7f4af29c1bdf3db5b81403, State: Initialized, Role: FOLLOWER
I20260924 21:16:55.621521 23178 consensus_queue.cc:260] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.621536 23214 ts_tablet_manager.cc:1905] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:55.621618 23178 ts_tablet_manager.cc:1428] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 21:16:55.623183 23214 ts_tablet_manager.cc:1918] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:16:55.623265 23214 log.cc:1196] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/2d36b7cd7ebd407b9ee9e15d40ed2693
I20260924 21:16:55.623505 23214 ts_tablet_manager.cc:1939] T 2d36b7cd7ebd407b9ee9e15d40ed2693 P 50c42c83fd7f4af29c1bdf3db5b81403: Deleting consensus metadata
I20260924 21:16:55.623891 22652 catalog_manager.cc:4841] TS 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673): tablet 2d36b7cd7ebd407b9ee9e15d40ed2693 (table test-table [id=b3581b7071ec4887971acb7466b4e837]) successfully deleted
I20260924 21:16:55.624432 23184 raft_consensus.cc:482] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:55.624496 23184 raft_consensus.cc:504] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.624626 23184 leader_election.cc:288] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673), 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305)
I20260924 21:16:55.624737 22825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd210bbb6e474c2d873ce26c1b1f692a" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" is_pre_election: true
I20260924 21:16:55.624835 22825 raft_consensus.cc:2454] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 0.
I20260924 21:16:55.624833 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd210bbb6e474c2d873ce26c1b1f692a" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422" is_pre_election: true
I20260924 21:16:55.624907 22965 raft_consensus.cc:2454] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 0.
I20260924 21:16:55.624966 23036 leader_election.cc:302] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c42c83fd7f4af29c1bdf3db5b81403, ad224586aa2343999b9a72f6cb305c5a; no voters:
I20260924 21:16:55.625123 23184 raft_consensus.cc:2781] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:55.625167 23184 raft_consensus.cc:482] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:55.625200 23184 raft_consensus.cc:3037] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:55.625689 23184 raft_consensus.cc:504] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.625823 23184 leader_election.cc:288] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 election: Requested vote from peers 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673), 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305)
I20260924 21:16:55.625937 22825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd210bbb6e474c2d873ce26c1b1f692a" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50c42c83fd7f4af29c1bdf3db5b81403"
I20260924 21:16:55.626019 22825 raft_consensus.cc:3037] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:55.625993 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd210bbb6e474c2d873ce26c1b1f692a" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422"
I20260924 21:16:55.626091 22965 raft_consensus.cc:3037] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:55.626565 22965 raft_consensus.cc:2454] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 1.
I20260924 21:16:55.626658 22825 raft_consensus.cc:2454] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 1.
I20260924 21:16:55.626716 23039 leader_election.cc:302] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6030eabf018a4316b6925487fac5b422, ad224586aa2343999b9a72f6cb305c5a; no voters:
I20260924 21:16:55.626794 23184 raft_consensus.cc:2781] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:55.626847 23184 raft_consensus.cc:686] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 1 LEADER]: Becoming Leader. State: Replica: ad224586aa2343999b9a72f6cb305c5a, State: Running, Role: LEADER
I20260924 21:16:55.626888 23184 consensus_queue.cc:237] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:55.627420 22667 catalog_manager.cc:5495] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a reported cstate change: term changed from 0 to 1, leader changed from <none> to ad224586aa2343999b9a72f6cb305c5a (127.20.115.195). New cstate: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260924 21:16:55.632733 22964 raft_consensus.cc:1261] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Refusing update from remote peer ad224586aa2343999b9a72f6cb305c5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:55.633045 23206 consensus_queue.cc:1035] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:55.633075 22825 raft_consensus.cc:1261] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Refusing update from remote peer ad224586aa2343999b9a72f6cb305c5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:55.633277 23206 consensus_queue.cc:1035] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 21:16:55.640260 23199 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 21:16:57.161492 22668 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:57.161590 22668 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:57.162487 22668 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47: Sending DeleteTablet for 3 replicas of tablet dd210bbb6e474c2d873ce26c1b1f692a
I20260924 21:16:57.162747 22805 tablet_service.cc:1514] Processing DeleteTablet for tablet dd210bbb6e474c2d873ce26c1b1f692a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:57 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:46828
I20260924 21:16:57.162760 22945 tablet_service.cc:1514] Processing DeleteTablet for tablet dd210bbb6e474c2d873ce26c1b1f692a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:57 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:41918
I20260924 21:16:57.162933 23085 tablet_service.cc:1514] Processing DeleteTablet for tablet dd210bbb6e474c2d873ce26c1b1f692a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:57 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:45572
I20260924 21:16:57.163115 23244 tablet_replica.cc:330] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: stopping tablet replica
I20260924 21:16:57.163195 23244 raft_consensus.cc:2229] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:57.163300 23244 raft_consensus.cc:2258] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:57.163582 23244 ts_tablet_manager.cc:1905] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:57.164539 23243 tablet_replica.cc:330] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: stopping tablet replica
I20260924 21:16:57.164608 23243 raft_consensus.cc:2229] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:57.164649 23243 raft_consensus.cc:2258] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:57.164752 23242 tablet_replica.cc:330] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: stopping tablet replica
I20260924 21:16:57.164827 23242 raft_consensus.cc:2229] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:57.164872 23242 raft_consensus.cc:2258] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:57.165108 23242 ts_tablet_manager.cc:1905] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:57.165729 23244 ts_tablet_manager.cc:1918] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:16:57.165798 23244 log.cc:1196] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/dd210bbb6e474c2d873ce26c1b1f692a
I20260924 21:16:57.166049 23244 ts_tablet_manager.cc:1939] T dd210bbb6e474c2d873ce26c1b1f692a P ad224586aa2343999b9a72f6cb305c5a: Deleting consensus metadata
I20260924 21:16:57.166618 22653 catalog_manager.cc:4841] TS ad224586aa2343999b9a72f6cb305c5a (127.20.115.195:39861): tablet dd210bbb6e474c2d873ce26c1b1f692a (table test-table [id=a7be71e76c924f42b6c83fddcf24ef6a]) successfully deleted
I20260924 21:16:57.166823 23242 ts_tablet_manager.cc:1918] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:16:57.166886 23242 log.cc:1196] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/dd210bbb6e474c2d873ce26c1b1f692a
I20260924 21:16:57.167093 23242 ts_tablet_manager.cc:1939] T dd210bbb6e474c2d873ce26c1b1f692a P 50c42c83fd7f4af29c1bdf3db5b81403: Deleting consensus metadata
I20260924 21:16:57.167325 23243 ts_tablet_manager.cc:1905] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:57.167423 22652 catalog_manager.cc:4841] TS 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673): tablet dd210bbb6e474c2d873ce26c1b1f692a (table test-table [id=a7be71e76c924f42b6c83fddcf24ef6a]) successfully deleted
I20260924 21:16:57.169059 23243 ts_tablet_manager.cc:1918] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:16:57.169121 23243 log.cc:1196] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/dd210bbb6e474c2d873ce26c1b1f692a
I20260924 21:16:57.169330 23243 ts_tablet_manager.cc:1939] T dd210bbb6e474c2d873ce26c1b1f692a P 6030eabf018a4316b6925487fac5b422: Deleting consensus metadata
I20260924 21:16:57.169703 22655 catalog_manager.cc:4841] TS 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305): tablet dd210bbb6e474c2d873ce26c1b1f692a (table test-table [id=a7be71e76c924f42b6c83fddcf24ef6a]) successfully deleted
Sep 24 21:16:57 dist-test-slave-cxpd krb5kdc[22623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:16:57 dist-test-slave-cxpd krb5kdc[22623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284617, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:16:57.190873 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.180404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54748 (local address 127.20.115.254:39003)
0924 21:16:57.180601 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:57.180605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:57.180619 (+ 14us) server_negotiation.cc:407] Connection header received
0924 21:16:57.180642 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:57.180643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:57.180653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:57.180713 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:16:57.181316 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.181912 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.182646 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.182841 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.182912 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.188569 (+ 5657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:16:57.188571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:16:57.188571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:16:57.188578 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:16:57.189914 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:57.190223 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:57.190223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:57.190224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:57.190272 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:16:57.190432 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:16:57.190433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:16:57.190433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:16:57.190542 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:16:57.190615 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:57.190715 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 21:16:57.190779 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":113,"threads_started":1}
I20260924 21:16:57.192237 22668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:16:57.192400 22668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:57.193672 22805 tablet_service.cc:1467] Processing CreateTablet for tablet 461eb0c50c7349d2a03ffd15cdd20e22 (DEFAULT_TABLE table=test-table [id=32805b992c9e4864b1f5dd7e9fb1f8d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:57.193701 22945 tablet_service.cc:1467] Processing CreateTablet for tablet 461eb0c50c7349d2a03ffd15cdd20e22 (DEFAULT_TABLE table=test-table [id=32805b992c9e4864b1f5dd7e9fb1f8d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:57.193722 23085 tablet_service.cc:1467] Processing CreateTablet for tablet 461eb0c50c7349d2a03ffd15cdd20e22 (DEFAULT_TABLE table=test-table [id=32805b992c9e4864b1f5dd7e9fb1f8d7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:57.193810 22945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461eb0c50c7349d2a03ffd15cdd20e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:57.193811 22805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461eb0c50c7349d2a03ffd15cdd20e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:57.193858 23085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461eb0c50c7349d2a03ffd15cdd20e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:57.194870 23253 tablet_bootstrap.cc:492] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: Bootstrap starting.
I20260924 21:16:57.195227 23254 tablet_bootstrap.cc:492] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: Bootstrap starting.
I20260924 21:16:57.195315 23253 tablet_bootstrap.cc:654] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:57.195559 23254 tablet_bootstrap.cc:654] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:57.195837 23255 tablet_bootstrap.cc:492] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: Bootstrap starting.
I20260924 21:16:57.195976 23253 tablet_bootstrap.cc:492] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: No bootstrap required, opened a new log
I20260924 21:16:57.196035 23253 ts_tablet_manager.cc:1397] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:57.196126 23254 tablet_bootstrap.cc:492] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: No bootstrap required, opened a new log
I20260924 21:16:57.196182 23254 ts_tablet_manager.cc:1397] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:57.196202 23255 tablet_bootstrap.cc:654] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:57.196162 23253 raft_consensus.cc:348] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.196245 23253 raft_consensus.cc:374] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:57.196280 23253 raft_consensus.cc:729] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad224586aa2343999b9a72f6cb305c5a, State: Initialized, Role: FOLLOWER
I20260924 21:16:57.196352 23253 consensus_queue.cc:260] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.196352 23254 raft_consensus.cc:348] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.196437 23254 raft_consensus.cc:374] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:57.196460 23254 raft_consensus.cc:729] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6030eabf018a4316b6925487fac5b422, State: Initialized, Role: FOLLOWER
I20260924 21:16:57.196528 23253 ts_tablet_manager.cc:1428] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:57.196527 23254 consensus_queue.cc:260] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.196699 23254 ts_tablet_manager.cc:1428] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:16:57.196739 23255 tablet_bootstrap.cc:492] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: No bootstrap required, opened a new log
I20260924 21:16:57.196801 23255 ts_tablet_manager.cc:1397] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:57.196945 23255 raft_consensus.cc:348] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.197006 23255 raft_consensus.cc:374] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:57.197023 23255 raft_consensus.cc:729] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c42c83fd7f4af29c1bdf3db5b81403, State: Initialized, Role: FOLLOWER
I20260924 21:16:57.197067 23255 consensus_queue.cc:260] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.197211 23255 ts_tablet_manager.cc:1428] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:57.389132 23225 raft_consensus.cc:482] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:57.389238 23225 raft_consensus.cc:504] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.389446 23225 leader_election.cc:288] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673), 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305)
I20260924 21:16:57.389704 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461eb0c50c7349d2a03ffd15cdd20e22" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422" is_pre_election: true
I20260924 21:16:57.389814 22965 raft_consensus.cc:2454] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 0.
I20260924 21:16:57.389765 22824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461eb0c50c7349d2a03ffd15cdd20e22" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" is_pre_election: true
I20260924 21:16:57.389895 22824 raft_consensus.cc:2454] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 0.
I20260924 21:16:57.390027 23039 leader_election.cc:302] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6030eabf018a4316b6925487fac5b422, ad224586aa2343999b9a72f6cb305c5a; no voters:
I20260924 21:16:57.390139 23225 raft_consensus.cc:2781] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:57.390183 23225 raft_consensus.cc:482] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:57.390208 23225 raft_consensus.cc:3037] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:57.390887 23225 raft_consensus.cc:504] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.391057 23225 leader_election.cc:288] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 election: Requested vote from peers 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673), 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305)
I20260924 21:16:57.391232 22824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461eb0c50c7349d2a03ffd15cdd20e22" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "50c42c83fd7f4af29c1bdf3db5b81403"
I20260924 21:16:57.391244 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461eb0c50c7349d2a03ffd15cdd20e22" candidate_uuid: "ad224586aa2343999b9a72f6cb305c5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422"
I20260924 21:16:57.391309 22824 raft_consensus.cc:3037] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:57.391330 22965 raft_consensus.cc:3037] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:57.391934 22824 raft_consensus.cc:2454] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 1.
I20260924 21:16:57.392047 22965 raft_consensus.cc:2454] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad224586aa2343999b9a72f6cb305c5a in term 1.
I20260924 21:16:57.392117 23036 leader_election.cc:302] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c42c83fd7f4af29c1bdf3db5b81403, ad224586aa2343999b9a72f6cb305c5a; no voters:
I20260924 21:16:57.392212 23225 raft_consensus.cc:2781] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:57.392256 23225 raft_consensus.cc:686] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 1 LEADER]: Becoming Leader. State: Replica: ad224586aa2343999b9a72f6cb305c5a, State: Running, Role: LEADER
I20260924 21:16:57.392302 23225 consensus_queue.cc:237] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.393013 22666 catalog_manager.cc:5495] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a reported cstate change: term changed from 0 to 1, leader changed from <none> to ad224586aa2343999b9a72f6cb305c5a (127.20.115.195). New cstate: current_term: 1 leader_uuid: "ad224586aa2343999b9a72f6cb305c5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } health_report { overall_health: HEALTHY } } }
I20260924 21:16:57.427498 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.425252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45594 (local address 127.20.115.195:39861)
0924 21:16:57.425398 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:57.425400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:57.425406 (+ 6us) server_negotiation.cc:407] Connection header received
0924 21:16:57.425424 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:57.425424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:57.425434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:57.425489 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:57.425649 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.426128 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.426866 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.427069 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.427128 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.427247 (+ 119us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:57.427272 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:57.427306 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:57.427366 (+ 60us) server_negotiation.cc:299] Negotiation successful
0924 21:16:57.427382 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260924 21:16:57.430852 22965 raft_consensus.cc:1261] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Refusing update from remote peer ad224586aa2343999b9a72f6cb305c5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:57.430893 22824 raft_consensus.cc:1261] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Refusing update from remote peer ad224586aa2343999b9a72f6cb305c5a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:57.431090 23230 consensus_queue.cc:1035] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:57.431500 23206 consensus_queue.cc:1035] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:57.450675 22666 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:57.450811 22666 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:57.451547 22666 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47: Sending DeleteTablet for 3 replicas of tablet 461eb0c50c7349d2a03ffd15cdd20e22
I20260924 21:16:57.451807 23085 tablet_service.cc:1514] Processing DeleteTablet for tablet 461eb0c50c7349d2a03ffd15cdd20e22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:57 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:45572
I20260924 21:16:57.451830 22805 tablet_service.cc:1514] Processing DeleteTablet for tablet 461eb0c50c7349d2a03ffd15cdd20e22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:57 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:46828
I20260924 21:16:57.451897 23244 tablet_replica.cc:330] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: stopping tablet replica
I20260924 21:16:57.451949 23244 raft_consensus.cc:2229] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:16:57.451954 23242 tablet_replica.cc:330] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: stopping tablet replica
I20260924 21:16:57.452005 23242 raft_consensus.cc:2229] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:57.452008 23244 raft_consensus.cc:2258] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:57.452052 23242 raft_consensus.cc:2258] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:57.452371 22945 tablet_service.cc:1514] Processing DeleteTablet for tablet 461eb0c50c7349d2a03ffd15cdd20e22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:57 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:41918
I20260924 21:16:57.452409 23244 ts_tablet_manager.cc:1905] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:57.452471 23243 tablet_replica.cc:330] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: stopping tablet replica
I20260924 21:16:57.452529 23243 raft_consensus.cc:2229] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:57.452579 23243 raft_consensus.cc:2258] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:57.452821 22666 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 21:16:57.452955 23243 ts_tablet_manager.cc:1905] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 21:16:57.452984 22666 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:16:57.453383 23242 ts_tablet_manager.cc:1905] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:57.454097 23244 ts_tablet_manager.cc:1918] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:16:57.454150 23244 log.cc:1196] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/461eb0c50c7349d2a03ffd15cdd20e22
I20260924 21:16:57.454290 22805 tablet_service.cc:1467] Processing CreateTablet for tablet a3b535a1f95d4c69b4d89c21f93181d4 (DEFAULT_TABLE table=test-table [id=941f8a2039f1438692af335f825ddd9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:57.454389 22805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3b535a1f95d4c69b4d89c21f93181d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:57.454380 23244 ts_tablet_manager.cc:1939] T 461eb0c50c7349d2a03ffd15cdd20e22 P ad224586aa2343999b9a72f6cb305c5a: Deleting consensus metadata
I20260924 21:16:57.454763 23243 ts_tablet_manager.cc:1918] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:16:57.454838 23243 log.cc:1196] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/461eb0c50c7349d2a03ffd15cdd20e22
I20260924 21:16:57.455123 23243 ts_tablet_manager.cc:1939] T 461eb0c50c7349d2a03ffd15cdd20e22 P 6030eabf018a4316b6925487fac5b422: Deleting consensus metadata
I20260924 21:16:57.455714 23255 tablet_bootstrap.cc:492] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403: Bootstrap starting.
I20260924 21:16:57.455818 22653 catalog_manager.cc:4841] TS ad224586aa2343999b9a72f6cb305c5a (127.20.115.195:39861): tablet 461eb0c50c7349d2a03ffd15cdd20e22 (table test-table [id=32805b992c9e4864b1f5dd7e9fb1f8d7]) successfully deleted
I20260924 21:16:57.455924 23085 tablet_service.cc:1467] Processing CreateTablet for tablet a3b535a1f95d4c69b4d89c21f93181d4 (DEFAULT_TABLE table=test-table [id=941f8a2039f1438692af335f825ddd9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:57.455997 23085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3b535a1f95d4c69b4d89c21f93181d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:57.456190 23255 tablet_bootstrap.cc:654] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:57.456918 23255 tablet_bootstrap.cc:492] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403: No bootstrap required, opened a new log
I20260924 21:16:57.456974 23255 ts_tablet_manager.cc:1397] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:57.457114 23255 raft_consensus.cc:348] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.457185 23255 raft_consensus.cc:374] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:57.457203 23255 raft_consensus.cc:729] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c42c83fd7f4af29c1bdf3db5b81403, State: Initialized, Role: FOLLOWER
I20260924 21:16:57.457192 23253 tablet_bootstrap.cc:492] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: Bootstrap starting.
I20260924 21:16:57.457248 23255 consensus_queue.cc:260] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.457327 23255 ts_tablet_manager.cc:1428] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:57.457396 22655 catalog_manager.cc:4841] TS 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305): tablet 461eb0c50c7349d2a03ffd15cdd20e22 (table test-table [id=32805b992c9e4864b1f5dd7e9fb1f8d7]) successfully deleted
I20260924 21:16:57.457525 23253 tablet_bootstrap.cc:654] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:57.457662 23242 ts_tablet_manager.cc:1918] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:16:57.457717 23242 log.cc:1196] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/461eb0c50c7349d2a03ffd15cdd20e22
I20260924 21:16:57.457940 23242 ts_tablet_manager.cc:1939] T 461eb0c50c7349d2a03ffd15cdd20e22 P 50c42c83fd7f4af29c1bdf3db5b81403: Deleting consensus metadata
I20260924 21:16:57.458103 22945 tablet_service.cc:1467] Processing CreateTablet for tablet a3b535a1f95d4c69b4d89c21f93181d4 (DEFAULT_TABLE table=test-table [id=941f8a2039f1438692af335f825ddd9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:16:57.458153 23253 tablet_bootstrap.cc:492] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: No bootstrap required, opened a new log
I20260924 21:16:57.458194 23253 ts_tablet_manager.cc:1397] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:57.458209 22945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3b535a1f95d4c69b4d89c21f93181d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:16:57.458307 23253 raft_consensus.cc:348] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.458377 23253 raft_consensus.cc:374] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:57.458400 23253 raft_consensus.cc:729] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad224586aa2343999b9a72f6cb305c5a, State: Initialized, Role: FOLLOWER
I20260924 21:16:57.458413 22652 catalog_manager.cc:4841] TS 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673): tablet 461eb0c50c7349d2a03ffd15cdd20e22 (table test-table [id=32805b992c9e4864b1f5dd7e9fb1f8d7]) successfully deleted
I20260924 21:16:57.458446 23253 consensus_queue.cc:260] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.458544 23253 ts_tablet_manager.cc:1428] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:57.459494 23254 tablet_bootstrap.cc:492] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: Bootstrap starting.
I20260924 21:16:57.459975 23254 tablet_bootstrap.cc:654] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: Neither blocks nor log segments found. Creating new log.
I20260924 21:16:57.460755 23254 tablet_bootstrap.cc:492] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: No bootstrap required, opened a new log
I20260924 21:16:57.460827 23254 ts_tablet_manager.cc:1397] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:16:57.460980 23254 raft_consensus.cc:348] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.461077 23254 raft_consensus.cc:374] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:16:57.461107 23254 raft_consensus.cc:729] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6030eabf018a4316b6925487fac5b422, State: Initialized, Role: FOLLOWER
I20260924 21:16:57.461153 23254 consensus_queue.cc:260] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.461259 23254 ts_tablet_manager.cc:1428] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:16:57.511787 23260 raft_consensus.cc:482] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:16:57.511890 23260 raft_consensus.cc:504] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.512326 23260 leader_election.cc:288] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305), ad224586aa2343999b9a72f6cb305c5a (127.20.115.195:39861)
I20260924 21:16:57.515432 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.512710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:34477 (local address 127.20.115.195:39861)
0924 21:16:57.512757 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:57.512759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:57.512859 (+ 100us) server_negotiation.cc:407] Connection header received
0924 21:16:57.512985 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:57.512986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:57.512996 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:57.513051 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:57.513195 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.513704 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.514684 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.515073 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.515157 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.515168 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:57.515325 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 21:16:57.515375 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:16:57.515456 23272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.512586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:39861 (local address 127.20.115.193:34477)
0924 21:16:57.512757 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:57.512767 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:57.512917 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:57.513054 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:57.513054 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:57.513056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:57.513119 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:57.513121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.513793 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.513795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:57.514591 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:57.514593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.515165 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.515166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:57.515241 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.515244 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:57.515317 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 21:16:57.515355 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":26,"thread_start_us":88,"threads_started":1}
I20260924 21:16:57.515554 23273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.512650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:51127 (local address 127.20.115.194:40305)
0924 21:16:57.512770 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:57.512772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:57.512922 (+ 150us) server_negotiation.cc:407] Connection header received
0924 21:16:57.513103 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:57.513104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:57.513112 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:57.513193 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:16:57.513363 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.513863 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.514763 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.515169 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.515259 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.515266 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:57.515420 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 21:16:57.515461 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260924 21:16:57.515772 23105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3b535a1f95d4c69b4d89c21f93181d4" candidate_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad224586aa2343999b9a72f6cb305c5a" is_pre_election: true
I20260924 21:16:57.515913 23105 raft_consensus.cc:2454] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c42c83fd7f4af29c1bdf3db5b81403 in term 0.
I20260924 21:16:57.516007 23271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.512586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40305 (local address 127.20.115.193:51127)
0924 21:16:57.512850 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:57.512859 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:57.512917 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:57.513217 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:57.513218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:57.513221 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:16:57.513300 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:57.513302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.513957 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.513958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:57.514671 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:57.514673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.515254 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.515255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:57.515324 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.515327 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:16:57.515413 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:16:57.515450 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":3,"thread_start_us":63,"threads_started":1}
I20260924 21:16:57.516114 22757 leader_election.cc:302] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c42c83fd7f4af29c1bdf3db5b81403, ad224586aa2343999b9a72f6cb305c5a; no voters:
I20260924 21:16:57.516196 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3b535a1f95d4c69b4d89c21f93181d4" candidate_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422" is_pre_election: true
I20260924 21:16:57.516242 23260 raft_consensus.cc:2781] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:16:57.516273 23260 raft_consensus.cc:482] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:16:57.516283 22965 raft_consensus.cc:2454] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50c42c83fd7f4af29c1bdf3db5b81403 in term 0.
I20260924 21:16:57.516294 23260 raft_consensus.cc:3037] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:57.517097 23260 raft_consensus.cc:504] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.517227 23260 leader_election.cc:288] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [CANDIDATE]: Term 1 election: Requested vote from peers 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305), ad224586aa2343999b9a72f6cb305c5a (127.20.115.195:39861)
I20260924 21:16:57.517388 23105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3b535a1f95d4c69b4d89c21f93181d4" candidate_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad224586aa2343999b9a72f6cb305c5a"
I20260924 21:16:57.517390 22965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3b535a1f95d4c69b4d89c21f93181d4" candidate_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6030eabf018a4316b6925487fac5b422"
I20260924 21:16:57.517467 22965 raft_consensus.cc:3037] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:57.517468 23105 raft_consensus.cc:3037] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:16:57.518043 22965 raft_consensus.cc:2454] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c42c83fd7f4af29c1bdf3db5b81403 in term 1.
I20260924 21:16:57.518121 23105 raft_consensus.cc:2454] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50c42c83fd7f4af29c1bdf3db5b81403 in term 1.
I20260924 21:16:57.518208 22759 leader_election.cc:302] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50c42c83fd7f4af29c1bdf3db5b81403, 6030eabf018a4316b6925487fac5b422; no voters:
I20260924 21:16:57.518311 23260 raft_consensus.cc:2781] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:16:57.518455 23260 raft_consensus.cc:686] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [term 1 LEADER]: Becoming Leader. State: Replica: 50c42c83fd7f4af29c1bdf3db5b81403, State: Running, Role: LEADER
I20260924 21:16:57.518533 23260 consensus_queue.cc:237] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } }
I20260924 21:16:57.519198 22667 catalog_manager.cc:5495] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50c42c83fd7f4af29c1bdf3db5b81403" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 43673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 } health_report { overall_health: UNKNOWN } } }
I20260924 21:16:57.538172 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.535672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.20.115.193:43673)
0924 21:16:57.535870 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:57.535872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:57.535879 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:16:57.535927 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:57.535928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:57.535948 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:57.536018 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:57.536127 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.536620 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.537379 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.537584 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.537638 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.537817 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:57.537917 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:57.537973 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:57.538043 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 21:16:57.538057 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":115,"threads_started":1}
I20260924 21:16:57.542565 23263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.540459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:36967 (local address 127.20.115.195:39861)
0924 21:16:57.540502 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:57.540504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:57.540598 (+ 94us) server_negotiation.cc:407] Connection header received
0924 21:16:57.540634 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:57.540635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:57.540645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:57.540690 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:57.540770 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.541268 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.541994 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.542178 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.542237 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.542372 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:57.542398 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:57.542437 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:57.542495 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 21:16:57.542513 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:16:57.542588 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.540389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:39861 (local address 127.20.115.193:36967)
0924 21:16:57.540527 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:57.540537 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:57.540596 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:57.540691 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:57.540691 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:57.540692 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:16:57.540733 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:57.540734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.541338 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.541339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:57.541907 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:57.541909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.542245 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.542246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:57.542292 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.542303 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:16:57.542456 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:16:57.542458 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:16:57.542492 (+ 34us) client_negotiation.cc:240] Negotiation successful
0924 21:16:57.542523 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260924 21:16:57.544664 22965 raft_consensus.cc:1261] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Refusing update from remote peer 50c42c83fd7f4af29c1bdf3db5b81403: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:57.544826 23105 raft_consensus.cc:1261] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Refusing update from remote peer 50c42c83fd7f4af29c1bdf3db5b81403: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:16:57.544925 23260 consensus_queue.cc:1035] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6030eabf018a4316b6925487fac5b422" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:57.545105 23274 consensus_queue.cc:1035] T a3b535a1f95d4c69b4d89c21f93181d4 P 50c42c83fd7f4af29c1bdf3db5b81403 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad224586aa2343999b9a72f6cb305c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 39861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:16:57.552512 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.550225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:45537 (local address 127.20.115.193:43673)
0924 21:16:57.550265 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:16:57.550266 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:16:57.550281 (+ 15us) server_negotiation.cc:407] Connection header received
0924 21:16:57.550325 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:16:57.550325 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:16:57.550336 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:16:57.550395 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:16:57.550562 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.551011 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.551767 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.552092 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.552153 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.552332 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:16:57.552359 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:16:57.552395 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0924 21:16:57.552445 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 21:16:57.552463 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:16:57.552538 23284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:57.550052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:43673 (local address 127.20.115.195:45537)
0924 21:16:57.550211 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:57.550223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:16:57.550277 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:16:57.550398 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:16:57.550399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:16:57.550401 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:16:57.550449 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:57.550451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.551089 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.551090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:57.551645 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:16:57.551647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:16:57.552161 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:16:57.552162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:16:57.552225 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:16:57.552230 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:16:57.552410 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:16:57.552411 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:16:57.552442 (+ 31us) client_negotiation.cc:240] Negotiation successful
0924 21:16:57.552482 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
W20260924 21:16:57.552906 23226 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260924 21:16:59.072098 22667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:59.072197 22667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:16:59.073151 22667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 888d0733c28a4d68a90d09b03c73dc47: Sending DeleteTablet for 3 replicas of tablet a3b535a1f95d4c69b4d89c21f93181d4
I20260924 21:16:59.073490 23085 tablet_service.cc:1514] Processing DeleteTablet for tablet a3b535a1f95d4c69b4d89c21f93181d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:59 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:45572
I20260924 21:16:59.073550 22805 tablet_service.cc:1514] Processing DeleteTablet for tablet a3b535a1f95d4c69b4d89c21f93181d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:59 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:46828
I20260924 21:16:59.073678 22945 tablet_service.cc:1514] Processing DeleteTablet for tablet a3b535a1f95d4c69b4d89c21f93181d4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:16:59 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:41918
I20260924 21:16:59.073689 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 22740
I20260924 21:16:59.074596 23309 tablet_replica.cc:330] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: stopping tablet replica
I20260924 21:16:59.074676 23309 raft_consensus.cc:2229] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:59.074728 23309 raft_consensus.cc:2258] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:59.074831 23311 tablet_replica.cc:330] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: stopping tablet replica
I20260924 21:16:59.074885 23311 raft_consensus.cc:2229] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:16:59.074924 23311 raft_consensus.cc:2258] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:16:59.075161 23311 ts_tablet_manager.cc:1905] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:59.075562 23309 ts_tablet_manager.cc:1905] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:16:59.076974 23311 ts_tablet_manager.cc:1918] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:16:59.077275 23309 ts_tablet_manager.cc:1918] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260924 21:16:59.077363 23311 log.cc:1196] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/a3b535a1f95d4c69b4d89c21f93181d4
I20260924 21:16:59.077354 23309 log.cc:1196] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/a3b535a1f95d4c69b4d89c21f93181d4
I20260924 21:16:59.077608 23309 ts_tablet_manager.cc:1939] T a3b535a1f95d4c69b4d89c21f93181d4 P ad224586aa2343999b9a72f6cb305c5a: Deleting consensus metadata
I20260924 21:16:59.077678 23311 ts_tablet_manager.cc:1939] T a3b535a1f95d4c69b4d89c21f93181d4 P 6030eabf018a4316b6925487fac5b422: Deleting consensus metadata
I20260924 21:16:59.077998 22655 catalog_manager.cc:4841] TS 6030eabf018a4316b6925487fac5b422 (127.20.115.194:40305): tablet a3b535a1f95d4c69b4d89c21f93181d4 (table test-table [id=941f8a2039f1438692af335f825ddd9e]) successfully deleted
I20260924 21:16:59.078132 22653 catalog_manager.cc:4841] TS ad224586aa2343999b9a72f6cb305c5a (127.20.115.195:39861): tablet a3b535a1f95d4c69b4d89c21f93181d4 (table test-table [id=941f8a2039f1438692af335f825ddd9e]) successfully deleted
W20260924 21:16:59.082260 22652 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:43673 (error 108)
I20260924 21:16:59.082854 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 22880
I20260924 21:16:59.082921 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:16:59.082689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:43673 (local address 127.0.0.1:46856)
0924 21:16:59.082824 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 21:16:59.082876 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:43673: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
W20260924 21:16:59.084249 22652 catalog_manager.cc:4584] TS 50c42c83fd7f4af29c1bdf3db5b81403 (127.20.115.193:43673): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a3b535a1f95d4c69b4d89c21f93181d4: Network error: Client connection negotiation failed: client connection to 127.20.115.193:43673: connect: Connection refused (error 111)
I20260924 21:16:59.087546 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 23020
I20260924 21:16:59.093159 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 22636
2026-09-24T21:16:59Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5734 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:16:59 dist-test-slave-cxpd krb5kdc[23316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:16:59 dist-test-slave-cxpd krb5kdc[23316](info): set up 2 sockets
Sep 24 21:16:59 dist-test-slave-cxpd krb5kdc[23316](info): commencing operation
krb5kdc: starting...
W20260924 21:17:00.132452 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:17:00.248195 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46601
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:39719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:46601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:00.327224 23329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:00.327484 23329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:00.327525 23329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:00.328660 23329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:00.328732 23329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:00.328758 23329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:00.328807 23329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:00.328836 23329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:00.330294 23329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:39719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:46601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:00.330585 23329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:00.330808 23329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:00.335906 23337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:00.335906 23334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:00.335872 23335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:00.336241 23329 server_base.cc:1034] running on GCE node
I20260924 21:17:00.336473 23329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:00.336735 23329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:00.337931 23329 hybrid_clock.cc:648] HybridClock initialized: now 1790284620337913 us; error 28 us; skew 500 ppm
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284620, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:00.340344 23329 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:17:00.340822 23329 webserver.cc:460] Webserver started at http://127.20.115.254:42719/ using document root <none> and password file <none>
I20260924 21:17:00.341063 23329 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:00.341107 23329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:00.341195 23329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:00.342177 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "fc7131e2d059473da0a7d8408b6119ff"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "6463bce8a76e23923d533eb13facdde7"
server_key_iv: "2ac1e92d7d374cbf6af3a4f1d608c6c3"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.342519 23329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "fc7131e2d059473da0a7d8408b6119ff"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "6463bce8a76e23923d533eb13facdde7"
server_key_iv: "2ac1e92d7d374cbf6af3a4f1d608c6c3"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.343822 23329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:17:00.344543 23344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.344714 23329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:00.344846 23329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "fc7131e2d059473da0a7d8408b6119ff"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "6463bce8a76e23923d533eb13facdde7"
server_key_iv: "2ac1e92d7d374cbf6af3a4f1d608c6c3"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.344945 23329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:00.365765 23329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:00.367581 23329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:00.367743 23329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:00.371757 23329 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:46601
I20260924 21:17:00.371806 23396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:46601 every 8 connection(s)
I20260924 21:17:00.372176 23329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:00.372881 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 23329
I20260924 21:17:00.372994 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:00.373198 20943 external_mini_cluster.cc:1428] Setting key 4e4996c28d4409b81779149b1586f7cd
I20260924 21:17:00.373207 23397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:00.376462 23397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff: Bootstrap starting.
I20260924 21:17:00.377190 23397 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:00.377501 23397 log.cc:824] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:00.378208 23397 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff: No bootstrap required, opened a new log
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:00.380817 23397 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7131e2d059473da0a7d8408b6119ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46601 } }
I20260924 21:17:00.380998 23397 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:00.381049 23397 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc7131e2d059473da0a7d8408b6119ff, State: Initialized, Role: FOLLOWER
I20260924 21:17:00.381189 23397 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7131e2d059473da0a7d8408b6119ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46601 } }
I20260924 21:17:00.381282 23397 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:00.381317 23397 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:00.381371 23397 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:00.382122 23397 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7131e2d059473da0a7d8408b6119ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46601 } }
I20260924 21:17:00.382230 23397 leader_election.cc:302] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc7131e2d059473da0a7d8408b6119ff; no voters:
I20260924 21:17:00.382398 23397 leader_election.cc:288] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:00.382527 23402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:00.382656 23397 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:00.382674 23402 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [term 1 LEADER]: Becoming Leader. State: Replica: fc7131e2d059473da0a7d8408b6119ff, State: Running, Role: LEADER
I20260924 21:17:00.382781 23402 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7131e2d059473da0a7d8408b6119ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46601 } }
I20260924 21:17:00.383144 23404 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc7131e2d059473da0a7d8408b6119ff. Latest consensus state: current_term: 1 leader_uuid: "fc7131e2d059473da0a7d8408b6119ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7131e2d059473da0a7d8408b6119ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46601 } } }
I20260924 21:17:00.383141 23403 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc7131e2d059473da0a7d8408b6119ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc7131e2d059473da0a7d8408b6119ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46601 } } }
I20260924 21:17:00.383256 23403 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:00.383404 23404 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:00.383529 23411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:00.384009 23411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:00.384454 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:00.373761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33680 (local address 127.20.115.254:46601)
0924 21:17:00.373932 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:00.373941 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:00.373976 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:17:00.374361 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:00.374367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:00.374419 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:00.374528 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:00.375297 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.376154 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.377223 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.377716 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.377811 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:00.380522 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:00.380528 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:00.380528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:00.380541 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:00.382566 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:00.382988 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:00.382990 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:00.382991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:00.383095 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:00.383334 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:00.383335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:00.383336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:00.383659 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:00.383852 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:00.384002 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 21:17:00.384262 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
I20260924 21:17:00.385731 23411 catalog_manager.cc:1347] Generated new cluster ID: 9f84cf640d5b4f41a3c5d1a2d6773d48
I20260924 21:17:00.385780 23411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:00.402724 23411 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:00.403527 23411 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:00.417119 23411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc7131e2d059473da0a7d8408b6119ff: Generated new TSK 0
I20260924 21:17:00.417346 23411 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:17:00.447991 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46601
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:39719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:00.530503 23423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:00.530738 23423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:00.530776 23423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:00.531927 23423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:00.531999 23423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:00.532047 23423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:00.533489 23423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:39719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:00.533746 23423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:00.533973 23423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:00.539104 23431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:00.539146 23423 server_base.cc:1034] running on GCE node
W20260924 21:17:00.539119 23429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:00.539117 23428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:00.539499 23423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:00.539757 23423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:00.540920 23423 hybrid_clock.cc:648] HybridClock initialized: now 1790284620540897 us; error 35 us; skew 500 ppm
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284620, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:00.543320 23423 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:17:00.543785 23423 webserver.cc:460] Webserver started at http://127.20.115.193:40613/ using document root <none> and password file <none>
I20260924 21:17:00.544027 23423 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:00.544096 23423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:00.544239 23423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:00.545311 23423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "5fbcd1b3e8b1439aa4c96e72749e641b"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "18acf11839a18ab88d3dfabd2ea232d9"
server_key_iv: "e03695dd9d181447bb50dc778e503d18"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.545657 23423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "5fbcd1b3e8b1439aa4c96e72749e641b"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "18acf11839a18ab88d3dfabd2ea232d9"
server_key_iv: "e03695dd9d181447bb50dc778e503d18"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.546957 23423 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:00.547717 23438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.547959 23423 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:00.548067 23423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "5fbcd1b3e8b1439aa4c96e72749e641b"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "18acf11839a18ab88d3dfabd2ea232d9"
server_key_iv: "e03695dd9d181447bb50dc778e503d18"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.548148 23423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:00.557436 23423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:00.559089 23423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:00.559244 23423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:00.559453 23423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:00.559748 23423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:00.559803 23423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.559835 23423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:00.559862 23423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.565668 23423 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:34285
I20260924 21:17:00.565719 23551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:34285 every 8 connection(s)
I20260924 21:17:00.566012 23423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284620, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:00.572921 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 23423
I20260924 21:17:00.573002 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:00.573211 20943 external_mini_cluster.cc:1428] Setting key 3286db32138ba092a717d097048818f3
I20260924 21:17:00.575455 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:00.566773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:54415 (local address 127.20.115.254:46601)
0924 21:17:00.566840 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:00.566842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:00.567033 (+ 191us) server_negotiation.cc:407] Connection header received
0924 21:17:00.567506 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:00.567507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:00.567517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:00.567583 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:00.568465 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.569076 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.570010 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.570214 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.570281 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:00.572312 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:00.572313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:00.572314 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:00.572320 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:00.573811 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:00.574369 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:00.574370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:00.574370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:00.574423 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:00.574699 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:00.574700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:00.574700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:00.574806 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:00.574889 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:00.575214 (+ 325us) server_negotiation.cc:299] Negotiation successful
0924 21:17:00.575323 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:17:00.575598 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:00.566750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46601 (local address 127.20.115.193:54415)
0924 21:17:00.566907 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:00.566946 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:00.567397 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:00.567635 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:00.567638 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:00.568186 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:00.568362 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:00.568366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.569154 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.569155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:00.569919 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:00.569921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.570294 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.570295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:00.570392 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:00.570900 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:00.570908 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:00.572227 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:00.573955 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:00.573961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:00.573964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:00.574293 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:00.574493 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:00.574493 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:00.574494 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:00.574635 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:00.574881 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:00.574887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:00.575056 (+ 169us) client_negotiation.cc:769] Sending connection context
0924 21:17:00.575169 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:17:00.575340 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260924 21:17:00.576671 23552 heartbeater.cc:347] Connected to a master server at 127.20.115.254:46601
I20260924 21:17:00.576812 23552 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:00.576997 23552 heartbeater.cc:511] Master 127.20.115.254:46601 requested a full tablet report, sending...
I20260924 21:17:00.577509 23361 ts_manager.cc:194] Registered new tserver with Master: 5fbcd1b3e8b1439aa4c96e72749e641b (127.20.115.193:34285)
I20260924 21:17:00.578126 23361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:54415
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:17:00.599730 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46601
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:39719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:00.683728 23557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:00.683974 23557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:00.684005 23557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:00.686134 23557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:00.686190 23557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:00.686228 23557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:00.688572 23557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:39719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:00.688822 23557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:00.689049 23557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:00.695477 23565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:00.695483 23563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:00.695799 23562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:00.696125 23557 server_base.cc:1034] running on GCE node
I20260924 21:17:00.696331 23557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:00.696576 23557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:00.697755 23557 hybrid_clock.cc:648] HybridClock initialized: now 1790284620697716 us; error 53 us; skew 500 ppm
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284620, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:00.701313 23557 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:17:00.701814 23557 webserver.cc:460] Webserver started at http://127.20.115.194:43103/ using document root <none> and password file <none>
I20260924 21:17:00.702087 23557 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:00.702162 23557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:00.702306 23557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:00.703874 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "b6b515411c1b43de980e905e3df1de63"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "7413acce2901d13a657d2e292e6fa682"
server_key_iv: "1f17e786709bc1cc772f41613e9c3343"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.704349 23557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "b6b515411c1b43de980e905e3df1de63"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "7413acce2901d13a657d2e292e6fa682"
server_key_iv: "1f17e786709bc1cc772f41613e9c3343"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.706259 23557 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260924 21:17:00.707248 23572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.707420 23557 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:00.707541 23557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "b6b515411c1b43de980e905e3df1de63"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "7413acce2901d13a657d2e292e6fa682"
server_key_iv: "1f17e786709bc1cc772f41613e9c3343"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.707625 23557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:00.718348 23557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:00.720166 23557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:00.720319 23557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:00.720532 23557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:00.720886 23557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:00.720923 23557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.720944 23557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:00.720986 23557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.726853 23557 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:40873
I20260924 21:17:00.726914 23685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:40873 every 8 connection(s)
I20260924 21:17:00.727196 23557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284620, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:00.734561 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 23557
I20260924 21:17:00.734664 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:00.734898 20943 external_mini_cluster.cc:1428] Setting key 5e3986e4032bfb104f57040304458ca8
I20260924 21:17:00.736368 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:00.727930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:55155 (local address 127.20.115.254:46601)
0924 21:17:00.727974 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:00.727976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:00.728104 (+ 128us) server_negotiation.cc:407] Connection header received
0924 21:17:00.728637 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:00.728638 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:00.728650 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:00.728708 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:00.729649 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.730172 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.731154 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.731349 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.731435 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:00.733471 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:00.733472 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:00.733472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:00.733479 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:00.734837 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:00.735387 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:00.735388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:00.735388 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:00.735452 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:00.735706 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:00.735707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:00.735707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:00.735820 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:00.735897 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:00.736195 (+ 298us) server_negotiation.cc:299] Negotiation successful
0924 21:17:00.736279 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:17:00.736552 23688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:00.727813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46601 (local address 127.20.115.194:55155)
0924 21:17:00.727989 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:00.728028 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:00.728530 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:00.728800 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:00.728804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:00.729369 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:00.729549 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:00.729553 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.730270 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.730271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:00.731061 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:00.731064 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.731446 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.731446 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:00.731568 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:00.732002 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:00.732009 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:00.733387 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:00.734979 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:00.734986 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:00.734988 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:00.735297 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:00.735522 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:00.735523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:00.735523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:00.735628 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:00.735889 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:00.735894 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:00.736035 (+ 141us) client_negotiation.cc:769] Sending connection context
0924 21:17:00.736160 (+ 125us) client_negotiation.cc:240] Negotiation successful
0924 21:17:00.736314 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260924 21:17:00.737299 23686 heartbeater.cc:347] Connected to a master server at 127.20.115.254:46601
I20260924 21:17:00.737404 23686 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:00.737605 23686 heartbeater.cc:511] Master 127.20.115.254:46601 requested a full tablet report, sending...
I20260924 21:17:00.737982 23361 ts_manager.cc:194] Registered new tserver with Master: b6b515411c1b43de980e905e3df1de63 (127.20.115.194:40873)
I20260924 21:17:00.738497 23361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:55155
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:17:00.760329 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46601
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:39719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:00.841681 23691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:00.841877 23691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:00.841900 23691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:00.843039 23691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:00.843078 23691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:00.843103 23691 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:00.844461 23691 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:39719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:00.844713 23691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:00.844950 23691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:00.849694 23697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:00.849677 23696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:00.849705 23691 server_base.cc:1034] running on GCE node
W20260924 21:17:00.849677 23699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:00.850139 23691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:00.850376 23691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:00.851534 23691 hybrid_clock.cc:648] HybridClock initialized: now 1790284620851515 us; error 33 us; skew 500 ppm
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284620, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:00.853894 23691 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:17:00.854313 23691 webserver.cc:460] Webserver started at http://127.20.115.195:35733/ using document root <none> and password file <none>
I20260924 21:17:00.854480 23691 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:00.854519 23691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:00.854605 23691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:00.855556 23691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "f0aa291339bc42fe96b37cb4803544ee"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "540ca1a82a2c30e0b34c14b8874bc9f3"
server_key_iv: "a11dd77b71d06972603fbb703fe6116d"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.855836 23691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "f0aa291339bc42fe96b37cb4803544ee"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "540ca1a82a2c30e0b34c14b8874bc9f3"
server_key_iv: "a11dd77b71d06972603fbb703fe6116d"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.857205 23691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:00.857867 23706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.858021 23691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:00.858098 23691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "f0aa291339bc42fe96b37cb4803544ee"
format_stamp: "Formatted at 2026-09-24 21:17:00 on dist-test-slave-cxpd"
server_key: "540ca1a82a2c30e0b34c14b8874bc9f3"
server_key_iv: "a11dd77b71d06972603fbb703fe6116d"
server_key_version: "encryptionkey@0"
I20260924 21:17:00.858147 23691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:00.882167 23691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:00.884876 23691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:00.884987 23691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:00.885152 23691 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:00.885393 23691 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:00.885423 23691 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.885453 23691 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:00.885466 23691 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:00.890901 23691 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:44783
I20260924 21:17:00.890975 23819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:44783 every 8 connection(s)
I20260924 21:17:00.891280 23691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:17:00.894773 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 23691
I20260924 21:17:00.894878 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:00.895066 20943 external_mini_cluster.cc:1428] Setting key 7e268b8200061aca99663e92ad61e3d9
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284620, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:00.900056 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:00.891960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:49911 (local address 127.20.115.254:46601)
0924 21:17:00.892074 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:00.892077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:00.892259 (+ 182us) server_negotiation.cc:407] Connection header received
0924 21:17:00.892837 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:00.892840 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:00.892853 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:00.892905 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:00.893844 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.894353 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.895564 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.895760 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.895855 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:00.897804 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:00.897806 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:00.897806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:00.897812 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:00.898953 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:00.899302 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:00.899303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:00.899303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:00.899351 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:00.899546 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:00.899547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:00.899547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:00.899651 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:00.899701 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:00.899936 (+ 235us) server_negotiation.cc:299] Negotiation successful
0924 21:17:00.899993 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 21:17:00.900193 23822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:00.891928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46601 (local address 127.20.115.195:49911)
0924 21:17:00.892127 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:00.892175 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:00.892674 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:00.892972 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:00.892976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:00.893558 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:00.893737 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:00.893741 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.894463 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.894464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:00.895470 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:00.895472 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:00.895866 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:00.895867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:00.895997 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:00.896387 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:00.896395 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:00.897726 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:00.899020 (+ 1294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:00.899025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:00.899026 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:00.899238 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:00.899403 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:00.899404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:00.899404 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:00.899492 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:00.899703 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:00.899706 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:00.899811 (+ 105us) client_negotiation.cc:769] Sending connection context
0924 21:17:00.899890 (+ 79us) client_negotiation.cc:240] Negotiation successful
0924 21:17:00.899998 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":88,"threads_started":1}
I20260924 21:17:00.900557 23820 heartbeater.cc:347] Connected to a master server at 127.20.115.254:46601
I20260924 21:17:00.900645 23820 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:00.900771 23820 heartbeater.cc:511] Master 127.20.115.254:46601 requested a full tablet report, sending...
I20260924 21:17:00.901161 23361 ts_manager.cc:194] Registered new tserver with Master: f0aa291339bc42fe96b37cb4803544ee (127.20.115.195:44783)
I20260924 21:17:00.901487 23361 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:49911
I20260924 21:17:00.906490 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 21:17:00.917394 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:00.916347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.20.115.254:46601)
0924 21:17:00.916408 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:00.916410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:00.916549 (+ 139us) server_negotiation.cc:407] Connection header received
0924 21:17:00.916586 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:00.916587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:00.916596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:00.916647 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:00.917332 (+ 685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33696: BlockingRecv error: recv got EOF from 127.0.0.1:33696 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:17:00.918294 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 23423
I20260924 21:17:00.922886 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 23557
I20260924 21:17:00.927002 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 23691
I20260924 21:17:00.931058 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 23329
2026-09-24T21:17:00Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1836 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23832](info): set up 2 sockets
Sep 24 21:17:00 dist-test-slave-cxpd krb5kdc[23832](info): commencing operation
krb5kdc: starting...
W20260924 21:17:01.964448 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:02 dist-test-slave-cxpd krb5kdc[23832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:17:02.076210 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:41913
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:41913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:02.155836 23845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:02.156081 23845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:02.156123 23845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:02.157302 23845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:02.157367 23845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:02.157392 23845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:02.157423 23845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:02.157450 23845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:02.158941 23845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:41913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:41913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:02.159209 23845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:02.159430 23845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:02.164733 23850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:02.164713 23851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:02.164716 23853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:02.164746 23845 server_base.cc:1034] running on GCE node
I20260924 21:17:02.165215 23845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:02.165506 23845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:02.166678 23845 hybrid_clock.cc:648] HybridClock initialized: now 1790284622166665 us; error 44 us; skew 500 ppm
Sep 24 21:17:02 dist-test-slave-cxpd krb5kdc[23832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284622, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:02.169158 23845 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:17:02.169643 23845 webserver.cc:460] Webserver started at http://127.20.115.254:34113/ using document root <none> and password file <none>
I20260924 21:17:02.169864 23845 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:02.169934 23845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:02.170092 23845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:02.171128 23845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "1e3957468abe467d96f92570567822ae"
format_stamp: "Formatted at 2026-09-24 21:17:02 on dist-test-slave-cxpd"
server_key: "ce332a4a903e1de8d74fd8b7b7b00d60"
server_key_iv: "3412d7584f73ba826959419bf213c433"
server_key_version: "encryptionkey@0"
I20260924 21:17:02.171494 23845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "1e3957468abe467d96f92570567822ae"
format_stamp: "Formatted at 2026-09-24 21:17:02 on dist-test-slave-cxpd"
server_key: "ce332a4a903e1de8d74fd8b7b7b00d60"
server_key_iv: "3412d7584f73ba826959419bf213c433"
server_key_version: "encryptionkey@0"
I20260924 21:17:02.172819 23845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:17:02.173574 23860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:02.173761 23845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:02.173861 23845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "1e3957468abe467d96f92570567822ae"
format_stamp: "Formatted at 2026-09-24 21:17:02 on dist-test-slave-cxpd"
server_key: "ce332a4a903e1de8d74fd8b7b7b00d60"
server_key_iv: "3412d7584f73ba826959419bf213c433"
server_key_version: "encryptionkey@0"
I20260924 21:17:02.173949 23845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:02.184468 23845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:02.186350 23845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:02.186522 23845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:02.190618 23845 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:41913
I20260924 21:17:02.190647 23912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:41913 every 8 connection(s)
I20260924 21:17:02.191120 23845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:02.191747 23913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:02.194000 23913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae: Bootstrap starting.
I20260924 21:17:02.194523 23913 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:02.194753 23913 log.cc:824] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:02.195372 23913 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae: No bootstrap required, opened a new log
I20260924 21:17:02.196941 23913 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3957468abe467d96f92570567822ae" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41913 } }
I20260924 21:17:02.197073 23913 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:02.197094 23913 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e3957468abe467d96f92570567822ae, State: Initialized, Role: FOLLOWER
I20260924 21:17:02.197227 23913 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3957468abe467d96f92570567822ae" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41913 } }
I20260924 21:17:02.197309 23913 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:02.197331 23913 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:02.197365 23913 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:02.198174 23913 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3957468abe467d96f92570567822ae" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41913 } }
I20260924 21:17:02.198256 23913 leader_election.cc:302] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e3957468abe467d96f92570567822ae; no voters:
I20260924 21:17:02.198365 23913 leader_election.cc:288] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:02.198448 23916 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:02.198616 23916 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [term 1 LEADER]: Becoming Leader. State: Replica: 1e3957468abe467d96f92570567822ae, State: Running, Role: LEADER
I20260924 21:17:02.198619 23913 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:02.198711 23916 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3957468abe467d96f92570567822ae" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41913 } }
I20260924 21:17:02.199057 23917 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e3957468abe467d96f92570567822ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3957468abe467d96f92570567822ae" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41913 } } }
I20260924 21:17:02.199065 23918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e3957468abe467d96f92570567822ae. Latest consensus state: current_term: 1 leader_uuid: "1e3957468abe467d96f92570567822ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e3957468abe467d96f92570567822ae" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41913 } } }
I20260924 21:17:02.199152 23917 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:02.199159 23918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:02.199493 23925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:02.199983 23925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:02.200395 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 23845
I20260924 21:17:02.200506 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:02.200683 20943 external_mini_cluster.cc:1428] Setting key e4190060ba1437c2fd65f29d9d9a274a
I20260924 21:17:02.201620 23925 catalog_manager.cc:1347] Generated new cluster ID: f6cb3b2cd2684eddbbf4ef672c2e50e8
I20260924 21:17:02.201674 23925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:17:02 dist-test-slave-cxpd krb5kdc[23832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:02.215761 23936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:02.201146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.20.115.254:41913)
0924 21:17:02.201324 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:02.201332 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:02.201781 (+ 449us) server_negotiation.cc:407] Connection header received
0924 21:17:02.201944 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:02.201949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:02.201993 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:02.202127 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:02.202742 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:02.203694 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:02.204561 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:02.205074 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:02.205210 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:02.212518 (+ 7308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:02.212522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:02.212522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:02.212534 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:02.214196 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:02.214585 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:02.214588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:02.214588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:02.214679 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:02.214891 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:02.214893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:02.214893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:02.215136 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:02.215274 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:02.215392 (+ 118us) server_negotiation.cc:299] Negotiation successful
0924 21:17:02.215548 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260924 21:17:02.216759 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:17:02.232057 23925 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:02.232684 23925 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:02.233609 23936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:02.229691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43126 (local address 127.20.115.254:41913)
0924 21:17:02.229762 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:02.229764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:02.229919 (+ 155us) server_negotiation.cc:407] Connection header received
0924 21:17:02.229982 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:02.229983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:02.229994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:02.230053 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:02.230482 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:02.231151 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:02.232036 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:02.232315 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:02.232381 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:02.233140 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:02.233142 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:02.233142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:02.233348 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:02.233365 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:02.233372 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:02.233427 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:02.233501 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 21:17:02.233539 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:17:02.236948 23925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e3957468abe467d96f92570567822ae: Generated new TSK 0
I20260924 21:17:02.237120 23925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:17:02.239197 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 23845
2026-09-24T21:17:02Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1306 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:02 dist-test-slave-cxpd krb5kdc[23946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:02 dist-test-slave-cxpd krb5kdc[23946](info): set up 2 sockets
Sep 24 21:17:02 dist-test-slave-cxpd krb5kdc[23946](info): commencing operation
krb5kdc: starting...
W20260924 21:17:03.264542 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.008s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:03 dist-test-slave-cxpd krb5kdc[23946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:17:03.379833 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:41359
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:41359
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:03.460031 23959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:03.460264 23959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:03.460300 23959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:03.461489 23959 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:03.461553 23959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:03.461587 23959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:03.461638 23959 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:03.461668 23959 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:03.463096 23959 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:41359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:41359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:03.463356 23959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:03.463570 23959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:03.468117 23964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:03.468117 23967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:03.468227 23965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:03.468420 23959 server_base.cc:1034] running on GCE node
I20260924 21:17:03.468673 23959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:03.469014 23959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:03.470184 23959 hybrid_clock.cc:648] HybridClock initialized: now 1790284623470168 us; error 26 us; skew 500 ppm
Sep 24 21:17:03 dist-test-slave-cxpd krb5kdc[23946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284623, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:03.472453 23959 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:17:03.472949 23959 webserver.cc:460] Webserver started at http://127.20.115.254:46651/ using document root <none> and password file <none>
I20260924 21:17:03.473171 23959 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:03.473237 23959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:03.473364 23959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:03.474364 23959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "51583023747447b59c6fe05c8ad0a12c"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "87916cab6f13d7dd2c530ec089c286fe"
server_key_iv: "2824f8af5a5c2b1437c49a8df3cdfda7"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.474712 23959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "51583023747447b59c6fe05c8ad0a12c"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "87916cab6f13d7dd2c530ec089c286fe"
server_key_iv: "2824f8af5a5c2b1437c49a8df3cdfda7"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.475991 23959 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:03.476666 23974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:03.476879 23959 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:03.476975 23959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "51583023747447b59c6fe05c8ad0a12c"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "87916cab6f13d7dd2c530ec089c286fe"
server_key_iv: "2824f8af5a5c2b1437c49a8df3cdfda7"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.477061 23959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:03.494206 23959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:03.495988 23959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:03.496150 23959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:03.500031 23959 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:41359
I20260924 21:17:03.500064 24026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:41359 every 8 connection(s)
I20260924 21:17:03.500379 23959 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:03.501282 24027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:03.503314 24027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c: Bootstrap starting.
I20260924 21:17:03.503813 24027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:03.504032 24027 log.cc:824] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:03.504259 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 23959
I20260924 21:17:03.504396 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:03.504606 20943 external_mini_cluster.cc:1428] Setting key adbb46814539fdf7067924eaa3e8acd4
I20260924 21:17:03.504896 24027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c: No bootstrap required, opened a new log
I20260924 21:17:03.506578 24027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51583023747447b59c6fe05c8ad0a12c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41359 } }
I20260924 21:17:03.506702 24027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:03.506724 24027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51583023747447b59c6fe05c8ad0a12c, State: Initialized, Role: FOLLOWER
I20260924 21:17:03.506879 24027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51583023747447b59c6fe05c8ad0a12c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41359 } }
I20260924 21:17:03.506968 24027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:03.507001 24027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:03.507050 24027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:03.508031 24027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51583023747447b59c6fe05c8ad0a12c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41359 } }
I20260924 21:17:03.508116 24027 leader_election.cc:302] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51583023747447b59c6fe05c8ad0a12c; no voters:
I20260924 21:17:03.508306 24027 leader_election.cc:288] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:03.508323 24032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:03.508430 24032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [term 1 LEADER]: Becoming Leader. State: Replica: 51583023747447b59c6fe05c8ad0a12c, State: Running, Role: LEADER
I20260924 21:17:03.508508 24032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51583023747447b59c6fe05c8ad0a12c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41359 } }
I20260924 21:17:03.508566 24027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:03.508828 24033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51583023747447b59c6fe05c8ad0a12c. Latest consensus state: current_term: 1 leader_uuid: "51583023747447b59c6fe05c8ad0a12c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51583023747447b59c6fe05c8ad0a12c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41359 } } }
I20260924 21:17:03.509052 24033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:03.508988 24034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51583023747447b59c6fe05c8ad0a12c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51583023747447b59c6fe05c8ad0a12c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41359 } } }
I20260924 21:17:03.509289 24034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:03.509598 24040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:03.510118 24040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:03.511556 24040 catalog_manager.cc:1347] Generated new cluster ID: aa867033a1fd4512b0f55608a53d67ac
I20260924 21:17:03.511611 24040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:17:03 dist-test-slave-cxpd krb5kdc[23946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:03.516845 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:03.505135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.20.115.254:41359)
0924 21:17:03.505322 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:03.505331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:03.505374 (+ 43us) server_negotiation.cc:407] Connection header received
0924 21:17:03.506574 (+ 1200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:03.506582 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:03.506636 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:03.506808 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:03.507591 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.508694 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.509850 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.510513 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.510615 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:03.513333 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:03.513337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:03.513338 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:03.513349 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:03.515192 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:03.515601 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:03.515604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:03.515604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:03.515717 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:03.515932 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:03.515933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:03.515933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:03.516196 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:03.516336 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:03.516517 (+ 181us) server_negotiation.cc:299] Negotiation successful
0924 21:17:03.516639 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260924 21:17:03.526664 24040 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:03.527343 24040 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:03.541320 24040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51583023747447b59c6fe05c8ad0a12c: Generated new TSK 0
I20260924 21:17:03.541509 24040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:17:03.573179 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:41359
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:03.653295 24053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:03.653484 24053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:03.653507 24053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:03.654556 24053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:03.654592 24053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:03.654616 24053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:03.656025 24053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:41359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:03.656206 24053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:03.656368 24053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:03.661253 24058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:03.661268 24061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:03.661391 24053 server_base.cc:1034] running on GCE node
W20260924 21:17:03.661269 24059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:03.661684 24053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:03.661913 24053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:03.663092 24053 hybrid_clock.cc:648] HybridClock initialized: now 1790284623663078 us; error 27 us; skew 500 ppm
Sep 24 21:17:03 dist-test-slave-cxpd krb5kdc[23946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284623, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:03.665598 24053 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:17:03.666087 24053 webserver.cc:460] Webserver started at http://127.20.115.193:36995/ using document root <none> and password file <none>
I20260924 21:17:03.666304 24053 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:03.666371 24053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:03.666523 24053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:03.667600 24053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "21c02a2f4faa46518f019db81df49cf5"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "d14348498767c15bed642764b53a30a6"
server_key_iv: "08c3b394c2e240961b43229e68b03ed2"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.667966 24053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "21c02a2f4faa46518f019db81df49cf5"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "d14348498767c15bed642764b53a30a6"
server_key_iv: "08c3b394c2e240961b43229e68b03ed2"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.669342 24053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:17:03.670078 24068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:03.670243 24053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:03.670344 24053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "21c02a2f4faa46518f019db81df49cf5"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "d14348498767c15bed642764b53a30a6"
server_key_iv: "08c3b394c2e240961b43229e68b03ed2"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.670420 24053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:03.693900 24053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:03.695940 24053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:03.696098 24053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:03.696331 24053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:03.696635 24053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:03.696688 24053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:03.696730 24053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:03.696758 24053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:03.705693 24053 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:44833
I20260924 21:17:03.705737 24181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:44833 every 8 connection(s)
I20260924 21:17:03.706077 24053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:17:03.707953 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 24053
I20260924 21:17:03.708042 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:03.708261 20943 external_mini_cluster.cc:1428] Setting key fb696263ad4deb71c74e0d4e9f101a8c
Sep 24 21:17:03 dist-test-slave-cxpd krb5kdc[23946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284623, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
I20260924 21:17:03.716599 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:03.706869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:42633 (local address 127.20.115.254:41359)
0924 21:17:03.706956 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:03.706958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:03.707162 (+ 204us) server_negotiation.cc:407] Connection header received
0924 21:17:03.707708 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:03.707709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:03.707725 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:03.707771 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:03.708986 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.709609 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.710760 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.710974 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.711062 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:03.713640 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:03.713642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:03.713642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:03.713650 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:03.715154 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:03.715602 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:03.715604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:03.715604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:03.715668 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:03.715938 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:03.715939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:03.715939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:03.716060 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:03.716140 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:03.716420 (+ 280us) server_negotiation.cc:299] Negotiation successful
0924 21:17:03.716509 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 21:17:03.716879 24184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:03.706831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41359 (local address 127.20.115.193:42633)
0924 21:17:03.707033 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:03.707069 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:03.707595 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:03.707846 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:03.707850 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:03.708620 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:03.708887 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:03.708892 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.709705 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.709706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:03.710671 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:03.710673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.711073 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.711074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:03.711236 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:03.711722 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:03.711732 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:03.713533 (+ 1801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:03.715269 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:03.715276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:03.715279 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:03.715511 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:03.715745 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:03.715746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:03.715747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:03.715855 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:03.716130 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:03.716136 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:03.716280 (+ 144us) client_negotiation.cc:769] Sending connection context
0924 21:17:03.716391 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 21:17:03.716550 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260924 21:17:03.717391 24182 heartbeater.cc:347] Connected to a master server at 127.20.115.254:41359
I20260924 21:17:03.717525 24182 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:03.717721 24182 heartbeater.cc:511] Master 127.20.115.254:41359 requested a full tablet report, sending...
I20260924 21:17:03.718302 23991 ts_manager.cc:194] Registered new tserver with Master: 21c02a2f4faa46518f019db81df49cf5 (127.20.115.193:44833)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
I20260924 21:17:03.719062 23991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:42633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:17:03.734179 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:41359
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:03.813850 24187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:03.814054 24187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:03.814078 24187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:03.815460 24187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:03.815516 24187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:03.815546 24187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:03.817061 24187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:41359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:03.817304 24187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:03.817552 24187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:03.822470 24193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:03.822484 24195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:03.822504 24187 server_base.cc:1034] running on GCE node
W20260924 21:17:03.822484 24192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:03.822896 24187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:03.823137 24187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:03.824325 24187 hybrid_clock.cc:648] HybridClock initialized: now 1790284623824307 us; error 43 us; skew 500 ppm
Sep 24 21:17:03 dist-test-slave-cxpd krb5kdc[23946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284623, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:03.826957 24187 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:17:03.827333 24187 webserver.cc:460] Webserver started at http://127.20.115.194:45181/ using document root <none> and password file <none>
I20260924 21:17:03.827581 24187 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:03.827651 24187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:03.827804 24187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:03.829054 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "0c26e35b947f4376a3a423d00a429cb7"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "8ade3d5abfd7ed01971abd3021d73728"
server_key_iv: "7be9bafb48b95967f16786faf0ef23af"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.829412 24187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "0c26e35b947f4376a3a423d00a429cb7"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "8ade3d5abfd7ed01971abd3021d73728"
server_key_iv: "7be9bafb48b95967f16786faf0ef23af"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.830773 24187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:03.831504 24202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:03.831691 24187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:03.831792 24187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "0c26e35b947f4376a3a423d00a429cb7"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "8ade3d5abfd7ed01971abd3021d73728"
server_key_iv: "7be9bafb48b95967f16786faf0ef23af"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.831910 24187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:03.845472 24187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:03.847355 24187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:03.847512 24187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:03.847769 24187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:03.848076 24187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:03.848131 24187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:03.848173 24187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:03.848201 24187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:03.854046 24187 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:43635
I20260924 21:17:03.854092 24315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:43635 every 8 connection(s)
I20260924 21:17:03.854405 24187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:17:03.858851 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 24187
I20260924 21:17:03.858981 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:03.859174 20943 external_mini_cluster.cc:1428] Setting key a0f4177095fdc72bbd30971a0bfd1d02
Sep 24 21:17:03 dist-test-slave-cxpd krb5kdc[23946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284623, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:03.864645 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:03.855133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:51867 (local address 127.20.115.254:41359)
0924 21:17:03.855218 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:03.855220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:03.855388 (+ 168us) server_negotiation.cc:407] Connection header received
0924 21:17:03.855858 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:03.855865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:03.855876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:03.855931 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:03.856959 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.857452 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.858528 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.858788 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.858901 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:03.861546 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:03.861547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:03.861548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:03.861557 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:03.863081 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:03.863601 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:03.863602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:03.863603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:03.863653 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:03.863987 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:03.863987 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:03.863988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:03.864105 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:03.864163 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:03.864511 (+ 348us) server_negotiation.cc:299] Negotiation successful
0924 21:17:03.864581 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 21:17:03.865020 24318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:03.855089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41359 (local address 127.20.115.194:51867)
0924 21:17:03.855266 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:03.855305 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:03.855770 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:03.856023 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:03.856027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:03.856653 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:03.856876 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:03.856882 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.857529 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.857530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:03.858442 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:03.858445 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:03.858876 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:03.858876 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:03.859068 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:03.859671 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:03.859679 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:03.861454 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:03.863169 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:03.863176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:03.863179 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:03.863515 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:03.863724 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:03.863724 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:03.863725 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:03.863906 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:03.864180 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:03.864186 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:03.864362 (+ 176us) client_negotiation.cc:769] Sending connection context
0924 21:17:03.864495 (+ 133us) client_negotiation.cc:240] Negotiation successful
0924 21:17:03.864683 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":101,"threads_started":1}
I20260924 21:17:03.865548 24316 heartbeater.cc:347] Connected to a master server at 127.20.115.254:41359
I20260924 21:17:03.865648 24316 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:03.865849 24316 heartbeater.cc:511] Master 127.20.115.254:41359 requested a full tablet report, sending...
I20260924 21:17:03.866262 23991 ts_manager.cc:194] Registered new tserver with Master: 0c26e35b947f4376a3a423d00a429cb7 (127.20.115.194:43635)
I20260924 21:17:03.866644 23991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:51867
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:17:03.884135 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:41359
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:03.963327 24321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:03.963557 24321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:03.963593 24321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:03.964709 24321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:03.964780 24321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:03.964861 24321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:03.966602 24321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:41359
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:03.966895 24321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:03.967132 24321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:03.971946 24329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:03.971944 24326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:03.972026 24327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:03.972342 24321 server_base.cc:1034] running on GCE node
I20260924 21:17:03.972530 24321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:03.972718 24321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:03.973856 24321 hybrid_clock.cc:648] HybridClock initialized: now 1790284623973840 us; error 30 us; skew 500 ppm
Sep 24 21:17:03 dist-test-slave-cxpd krb5kdc[23946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284623, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:03.976293 24321 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:17:03.976749 24321 webserver.cc:460] Webserver started at http://127.20.115.195:42929/ using document root <none> and password file <none>
I20260924 21:17:03.976997 24321 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:03.977042 24321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:03.977129 24321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:03.978068 24321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "446419df1e5f43d083225ec06bfb51f2"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "4999abe753226ebe1f6f1adf26bf89a8"
server_key_iv: "2384a48c62e64fd703077de6f5473ed4"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.978410 24321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "446419df1e5f43d083225ec06bfb51f2"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "4999abe753226ebe1f6f1adf26bf89a8"
server_key_iv: "2384a48c62e64fd703077de6f5473ed4"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.979687 24321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:17:03.980358 24336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:03.980547 24321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:03.980616 24321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "446419df1e5f43d083225ec06bfb51f2"
format_stamp: "Formatted at 2026-09-24 21:17:03 on dist-test-slave-cxpd"
server_key: "4999abe753226ebe1f6f1adf26bf89a8"
server_key_iv: "2384a48c62e64fd703077de6f5473ed4"
server_key_version: "encryptionkey@0"
I20260924 21:17:03.980705 24321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:03.991497 24321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:03.993718 24321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:03.993861 24321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:03.994084 24321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:03.994370 24321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:03.994407 24321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:03.994463 24321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:03.994496 24321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:04.000214 24321 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:42969
I20260924 21:17:04.000276 24449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:42969 every 8 connection(s)
I20260924 21:17:04.000563 24321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:17:04 dist-test-slave-cxpd krb5kdc[23946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284623, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:04.008760 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 24321
I20260924 21:17:04.008912 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:04.009137 20943 external_mini_cluster.cc:1428] Setting key 63b381cd79084494354530f50c95a382
I20260924 21:17:04.010176 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:04.001273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:58667 (local address 127.20.115.254:41359)
0924 21:17:04.001312 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:04.001314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:04.001539 (+ 225us) server_negotiation.cc:407] Connection header received
0924 21:17:04.002063 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:04.002065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:04.002077 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:04.002128 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:04.003034 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:04.003531 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:04.004528 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:04.004843 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:04.004928 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:04.006889 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:04.006891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:04.006891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:04.006898 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:04.008300 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:04.008882 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:04.008884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:04.008884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:04.008956 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:04.009276 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:04.009277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:04.009277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:04.009395 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:04.009476 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:04.009993 (+ 517us) server_negotiation.cc:299] Negotiation successful
0924 21:17:04.010064 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:17:04.010347 24452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:04.001240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41359 (local address 127.20.115.195:58667)
0924 21:17:04.001407 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:04.001463 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:04.001949 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:04.002191 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:04.002195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:04.002740 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:04.002934 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:04.002938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:04.003626 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:04.003627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:04.004434 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:04.004436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:04.004931 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:04.004932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:04.005067 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:04.005476 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:04.005484 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:04.006799 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:04.008388 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:04.008393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:04.008394 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:04.008753 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:04.009063 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:04.009064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:04.009065 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:04.009206 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:04.009469 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:04.009475 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:04.009790 (+ 315us) client_negotiation.cc:769] Sending connection context
0924 21:17:04.009940 (+ 150us) client_negotiation.cc:240] Negotiation successful
0924 21:17:04.010107 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":85,"threads_started":1}
I20260924 21:17:04.010799 24450 heartbeater.cc:347] Connected to a master server at 127.20.115.254:41359
I20260924 21:17:04.010906 24450 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:04.011077 24450 heartbeater.cc:511] Master 127.20.115.254:41359 requested a full tablet report, sending...
I20260924 21:17:04.011428 23991 ts_manager.cc:194] Registered new tserver with Master: 446419df1e5f43d083225ec06bfb51f2 (127.20.115.195:42969)
I20260924 21:17:04.011808 23991 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:58667
I20260924 21:17:04.020946 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 21:17:04 dist-test-slave-cxpd krb5kdc[23946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284624, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 24 21:17:04 dist-test-slave-cxpd krb5kdc[23946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284624, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:04.042841 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:04.035686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53190 (local address 127.20.115.254:41359)
0924 21:17:04.035749 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:04.035751 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:04.035884 (+ 133us) server_negotiation.cc:407] Connection header received
0924 21:17:04.035977 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:04.035978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:04.035992 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:04.036054 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:04.036620 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:04.037170 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:04.037925 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:04.038121 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:04.038193 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:04.040314 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:04.040316 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:04.040316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:04.040324 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:04.041688 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:04.042049 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:04.042050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:04.042050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:04.042112 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:04.042303 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:04.042304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:04.042304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:04.042454 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:04.042565 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:04.042662 (+ 97us) server_negotiation.cc:299] Negotiation successful
0924 21:17:04.042740 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260924 21:17:04.043072 23991 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53190
I20260924 21:17:04.043458 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 24053
I20260924 21:17:04.047995 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 24187
I20260924 21:17:04.052150 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 24321
I20260924 21:17:04.056528 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 23959
2026-09-24T21:17:04Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1820 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:04 dist-test-slave-cxpd krb5kdc[24462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:04 dist-test-slave-cxpd krb5kdc[24462](info): set up 2 sockets
Sep 24 21:17:04 dist-test-slave-cxpd krb5kdc[24462](info): commencing operation
krb5kdc: starting...
W20260924 21:17:05.104703 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:17:05.223867 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45031
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:45031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:05.303174 24475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:05.303382 24475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:05.303406 24475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:05.304468 24475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:05.304502 24475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:05.304517 24475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:05.304528 24475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:05.304541 24475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:05.305977 24475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:45031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:05.306162 24475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:05.306319 24475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:05.311120 24480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:05.311113 24481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:05.311160 24475 server_base.cc:1034] running on GCE node
W20260924 21:17:05.311113 24483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:05.311592 24475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:05.311872 24475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:05.313028 24475 hybrid_clock.cc:648] HybridClock initialized: now 1790284625313016 us; error 30 us; skew 500 ppm
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:05.315541 24475 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:17:05.315955 24475 webserver.cc:460] Webserver started at http://127.20.115.254:38993/ using document root <none> and password file <none>
I20260924 21:17:05.316172 24475 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:05.316215 24475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:05.316368 24475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:05.317456 24475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "9b01059a4292485ab980cdea033d5afe"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "82a12348241bfed0ee9a14eed50d4664"
server_key_iv: "5c3c9457c8fb09dfc6c7c470b30b1461"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.317816 24475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "9b01059a4292485ab980cdea033d5afe"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "82a12348241bfed0ee9a14eed50d4664"
server_key_iv: "5c3c9457c8fb09dfc6c7c470b30b1461"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.319084 24475 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:05.319801 24490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.319989 24475 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:05.320057 24475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "9b01059a4292485ab980cdea033d5afe"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "82a12348241bfed0ee9a14eed50d4664"
server_key_iv: "5c3c9457c8fb09dfc6c7c470b30b1461"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.320144 24475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:05.348805 24475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:05.350628 24475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:05.350787 24475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:05.354730 24475 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:45031
I20260924 21:17:05.354768 24542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:45031 every 8 connection(s)
I20260924 21:17:05.355269 24475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:05.355890 24543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:05.358098 24543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe: Bootstrap starting.
I20260924 21:17:05.358686 24543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:05.358670 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 24475
I20260924 21:17:05.358770 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:05.358979 20943 external_mini_cluster.cc:1428] Setting key a88b09620e31d4fac4b03ec4ff276c4e
I20260924 21:17:05.359011 24543 log.cc:824] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:05.359752 24543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe: No bootstrap required, opened a new log
I20260924 21:17:05.362144 24543 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b01059a4292485ab980cdea033d5afe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45031 } }
I20260924 21:17:05.362308 24543 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:05.362341 24543 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b01059a4292485ab980cdea033d5afe, State: Initialized, Role: FOLLOWER
I20260924 21:17:05.362457 24543 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b01059a4292485ab980cdea033d5afe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45031 } }
I20260924 21:17:05.362531 24543 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:05.362565 24543 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:05.362607 24543 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:05.363188 24543 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b01059a4292485ab980cdea033d5afe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45031 } }
I20260924 21:17:05.363270 24543 leader_election.cc:302] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b01059a4292485ab980cdea033d5afe; no voters:
I20260924 21:17:05.363464 24543 leader_election.cc:288] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:05.363489 24548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:05.363605 24548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [term 1 LEADER]: Becoming Leader. State: Replica: 9b01059a4292485ab980cdea033d5afe, State: Running, Role: LEADER
I20260924 21:17:05.363699 24548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b01059a4292485ab980cdea033d5afe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45031 } }
I20260924 21:17:05.364034 24543 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:05.364001 24548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b01059a4292485ab980cdea033d5afe. Latest consensus state: current_term: 1 leader_uuid: "9b01059a4292485ab980cdea033d5afe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b01059a4292485ab980cdea033d5afe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45031 } } }
I20260924 21:17:05.364282 24548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:05.364012 24550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b01059a4292485ab980cdea033d5afe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b01059a4292485ab980cdea033d5afe" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45031 } } }
I20260924 21:17:05.364620 24550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:05.364892 24552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:05.365381 24552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:05.366912 24552 catalog_manager.cc:1347] Generated new cluster ID: 6e2e6c7c454b4414a0f4225ed8924db8
I20260924 21:17:05.366966 24552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:05.369848 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:05.359469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37478 (local address 127.20.115.254:45031)
0924 21:17:05.359625 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:05.359632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:05.359655 (+ 23us) server_negotiation.cc:407] Connection header received
0924 21:17:05.360098 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:05.360105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:05.360150 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:05.360259 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:05.361037 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.361855 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.362868 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.363283 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.363397 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:05.366272 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:05.366276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:05.366276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:05.366287 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:05.368164 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.368526 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.368530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.368530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.368618 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.368856 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.368857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.368858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.369088 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:05.369255 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:05.369590 (+ 335us) server_negotiation.cc:299] Negotiation successful
0924 21:17:05.369700 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260924 21:17:05.370949 24552 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:05.371624 24552 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:05.374426 24552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b01059a4292485ab980cdea033d5afe: Generated new TSK 0
I20260924 21:17:05.374583 24552 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:17:05.403813 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45031
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:05.494727 24569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:05.494982 24569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:05.495023 24569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:05.496176 24569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:05.496244 24569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:05.496289 24569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:05.497864 24569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:05.498124 24569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:05.498351 24569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:05.503343 24575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:05.503357 24574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:05.503482 24569 server_base.cc:1034] running on GCE node
W20260924 21:17:05.503348 24577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:05.503861 24569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:05.504145 24569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:05.505296 24569 hybrid_clock.cc:648] HybridClock initialized: now 1790284625505281 us; error 29 us; skew 500 ppm
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:05.507774 24569 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:17:05.508231 24569 webserver.cc:460] Webserver started at http://127.20.115.193:42643/ using document root <none> and password file <none>
I20260924 21:17:05.508451 24569 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:05.508519 24569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:05.508689 24569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:05.509768 24569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "fa9539494667475484f872dd5b980bfe"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "fc85b091822bc5ef2b8cabe22ccd0628"
server_key_iv: "b214d790ce50f517c79b0ea9c61b857c"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.510131 24569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "fa9539494667475484f872dd5b980bfe"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "fc85b091822bc5ef2b8cabe22ccd0628"
server_key_iv: "b214d790ce50f517c79b0ea9c61b857c"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.511543 24569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:05.512275 24584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.512452 24569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:05.512549 24569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "fa9539494667475484f872dd5b980bfe"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "fc85b091822bc5ef2b8cabe22ccd0628"
server_key_iv: "b214d790ce50f517c79b0ea9c61b857c"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.512634 24569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:05.527002 24569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:05.529520 24569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:05.529668 24569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:05.529955 24569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:05.530292 24569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:05.530344 24569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.530385 24569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:05.530412 24569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.536118 24569 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:33629
I20260924 21:17:05.536201 24697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:33629 every 8 connection(s)
I20260924 21:17:05.536522 24569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:17:05.539167 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 24569
I20260924 21:17:05.539280 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:05.539475 20943 external_mini_cluster.cc:1428] Setting key d6af9abba801efc501a681c806e72c02
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:05.546456 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:05.537265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:40381 (local address 127.20.115.254:45031)
0924 21:17:05.537328 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:05.537330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:05.537533 (+ 203us) server_negotiation.cc:407] Connection header received
0924 21:17:05.538473 (+ 940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:05.538475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:05.538486 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:05.538538 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:05.539284 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.539963 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.540961 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.541197 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.541286 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:05.543350 (+ 2064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:05.543352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:05.543353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:05.543361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:05.545030 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.545438 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.545439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.545440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.545503 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.545736 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.545737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.545737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.545876 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:05.545965 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:05.546267 (+ 302us) server_negotiation.cc:299] Negotiation successful
0924 21:17:05.546355 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:17:05.546550 24700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:05.537231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45031 (local address 127.20.115.193:40381)
0924 21:17:05.537416 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:05.537450 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:05.538359 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:05.538610 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:05.538615 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:05.538869 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:05.539152 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:05.539158 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.540057 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.540058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:05.540866 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:05.540868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.541297 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.541298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:05.541418 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:05.541936 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:05.541944 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:05.543260 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:05.545130 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:05.545136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:05.545137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:05.545358 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:05.545573 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:05.545574 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:05.545574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:05.545670 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:05.545996 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:05.546002 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:05.546121 (+ 119us) client_negotiation.cc:769] Sending connection context
0924 21:17:05.546220 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 21:17:05.546341 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260924 21:17:05.547050 24698 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45031
I20260924 21:17:05.547160 24698 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:05.547330 24698 heartbeater.cc:511] Master 127.20.115.254:45031 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
I20260924 21:17:05.547914 24507 ts_manager.cc:194] Registered new tserver with Master: fa9539494667475484f872dd5b980bfe (127.20.115.193:33629)
I20260924 21:17:05.548578 24507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:40381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:17:05.565364 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45031
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:05.645273 24703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:05.645467 24703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:05.645488 24703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:05.646564 24703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:05.646598 24703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:05.646623 24703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:05.648017 24703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:05.648192 24703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:05.648360 24703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:05.653164 24708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:05.653209 24709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:05.653164 24711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:05.653388 24703 server_base.cc:1034] running on GCE node
I20260924 21:17:05.653638 24703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:05.653880 24703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:05.655054 24703 hybrid_clock.cc:648] HybridClock initialized: now 1790284625655040 us; error 27 us; skew 500 ppm
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:05.657435 24703 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:17:05.657934 24703 webserver.cc:460] Webserver started at http://127.20.115.194:42667/ using document root <none> and password file <none>
I20260924 21:17:05.658167 24703 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:05.658236 24703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:05.658391 24703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:05.659436 24703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "9739aebde42a4ec1aecd5a33124c7c6a"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "c3f2331987e046e89e2f568f151c647a"
server_key_iv: "bd85f6f39677d21dcc952aa777352d6a"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.659814 24703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "9739aebde42a4ec1aecd5a33124c7c6a"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "c3f2331987e046e89e2f568f151c647a"
server_key_iv: "bd85f6f39677d21dcc952aa777352d6a"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.661213 24703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:05.661972 24718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.662185 24703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:05.662281 24703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "9739aebde42a4ec1aecd5a33124c7c6a"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "c3f2331987e046e89e2f568f151c647a"
server_key_iv: "bd85f6f39677d21dcc952aa777352d6a"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.662365 24703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:05.703794 24703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:05.705812 24703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:05.705996 24703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:05.706254 24703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:05.706568 24703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:05.706622 24703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.706660 24703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:05.706686 24703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.712461 24703 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:35763
I20260924 21:17:05.712513 24831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:35763 every 8 connection(s)
I20260924 21:17:05.712873 24703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:05.720273 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 24703
I20260924 21:17:05.720422 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:05.720613 20943 external_mini_cluster.cc:1428] Setting key e9d81933adca6cc2b4057ca53f364e50
I20260924 21:17:05.722416 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:05.713646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:57375 (local address 127.20.115.254:45031)
0924 21:17:05.713705 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:05.713708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:05.713930 (+ 222us) server_negotiation.cc:407] Connection header received
0924 21:17:05.714903 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:05.714904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:05.714914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:05.714978 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:05.715622 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.716103 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.717194 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.717377 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.717434 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:05.719336 (+ 1902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:05.719338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:05.719338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:05.719344 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:05.720913 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.721371 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.721372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.721372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.721438 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.721678 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.721679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.721680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.721819 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:05.721891 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:05.722201 (+ 310us) server_negotiation.cc:299] Negotiation successful
0924 21:17:05.722284 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:17:05.722554 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:05.713640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45031 (local address 127.20.115.194:57375)
0924 21:17:05.713797 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:05.713837 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:05.714784 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:05.715034 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:05.715038 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:05.715356 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:05.715546 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:05.715550 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.716188 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.716189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:05.717120 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:05.717123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.717451 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.717452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:05.717545 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:05.717966 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:05.717974 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:05.719257 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:05.720985 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:05.720991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:05.720993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:05.721276 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:05.721514 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:05.721515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:05.721515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:05.721615 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:05.721908 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:05.721912 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:05.722021 (+ 109us) client_negotiation.cc:769] Sending connection context
0924 21:17:05.722171 (+ 150us) client_negotiation.cc:240] Negotiation successful
0924 21:17:05.722291 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260924 21:17:05.723076 24832 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45031
I20260924 21:17:05.723191 24832 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:05.723373 24832 heartbeater.cc:511] Master 127.20.115.254:45031 requested a full tablet report, sending...
I20260924 21:17:05.723758 24507 ts_manager.cc:194] Registered new tserver with Master: 9739aebde42a4ec1aecd5a33124c7c6a (127.20.115.194:35763)
I20260924 21:17:05.724136 24507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:57375
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:17:05.745880 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45031
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:05.825505 24837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:05.825691 24837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:05.825713 24837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:05.827368 24837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:05.827440 24837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:05.827477 24837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:05.828953 24837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:05.829191 24837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:05.829414 24837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:05.837147 24842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:05.837340 24845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:05.837446 24837 server_base.cc:1034] running on GCE node
W20260924 21:17:05.837180 24843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:05.837731 24837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:05.838003 24837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:05.839156 24837 hybrid_clock.cc:648] HybridClock initialized: now 1790284625839140 us; error 29 us; skew 500 ppm
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:05.841749 24837 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:17:05.842159 24837 webserver.cc:460] Webserver started at http://127.20.115.195:42611/ using document root <none> and password file <none>
I20260924 21:17:05.842389 24837 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:05.842455 24837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:05.842604 24837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:05.843597 24837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "d5825f0fc3a34ea7b9cc9e8131181369"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "db394c102eff4c9b62fec3078d051029"
server_key_iv: "f7e826626452727db80c194f0c1f2109"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.843951 24837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "d5825f0fc3a34ea7b9cc9e8131181369"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "db394c102eff4c9b62fec3078d051029"
server_key_iv: "f7e826626452727db80c194f0c1f2109"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.845327 24837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:17:05.846076 24852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.846235 24837 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:05.846333 24837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "d5825f0fc3a34ea7b9cc9e8131181369"
format_stamp: "Formatted at 2026-09-24 21:17:05 on dist-test-slave-cxpd"
server_key: "db394c102eff4c9b62fec3078d051029"
server_key_iv: "f7e826626452727db80c194f0c1f2109"
server_key_version: "encryptionkey@0"
I20260924 21:17:05.846410 24837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:05.862643 24837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:05.864540 24837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:05.864754 24837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:05.865145 24837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:05.865458 24837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:05.865512 24837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.865546 24837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:05.865573 24837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:05.871433 24837 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:35005
I20260924 21:17:05.871510 24965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:35005 every 8 connection(s)
I20260924 21:17:05.871819 24837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:05.880779 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:05.872615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:52843 (local address 127.20.115.254:45031)
0924 21:17:05.872680 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:05.872681 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:05.872908 (+ 227us) server_negotiation.cc:407] Connection header received
0924 21:17:05.873821 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:05.873822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:05.873832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:05.873882 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:05.874537 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.875234 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.876197 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.876385 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.876445 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:05.878428 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:05.878429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:05.878429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:05.878436 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:05.879588 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.879947 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.879948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.879948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.880004 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.880205 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.880206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.880206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.880301 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:05.880349 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:05.880563 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 21:17:05.880673 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:17:05.880807 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 24837
I20260924 21:17:05.880888 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:05.880883 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:05.872599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45031 (local address 127.20.115.195:52843)
0924 21:17:05.872753 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:05.872831 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:05.873717 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:05.873947 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:05.873950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:05.874174 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:05.874434 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:05.874439 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.875316 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.875316 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:05.876113 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:05.876115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.876461 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.876462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:05.876554 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:05.877081 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:05.877088 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:05.878348 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:05.879658 (+ 1310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:05.879664 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:05.879665 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:05.879866 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:05.880062 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:05.880062 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:05.880062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:05.880152 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:05.880353 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:05.880357 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:05.880461 (+ 104us) client_negotiation.cc:769] Sending connection context
0924 21:17:05.880538 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 21:17:05.880646 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260924 21:17:05.881074 20943 external_mini_cluster.cc:1428] Setting key f113663a04d566b148d4e92da72f3a03
I20260924 21:17:05.881362 24966 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45031
I20260924 21:17:05.881474 24966 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:05.881651 24966 heartbeater.cc:511] Master 127.20.115.254:45031 requested a full tablet report, sending...
I20260924 21:17:05.882048 24507 ts_manager.cc:194] Registered new tserver with Master: d5825f0fc3a34ea7b9cc9e8131181369 (127.20.115.195:35005)
I20260924 21:17:05.882408 24507 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:52843
I20260924 21:17:05.892993 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
I20260924 21:17:05.915895 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:05.908122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53694 (local address 127.20.115.193:33629)
0924 21:17:05.908303 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:05.908308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:05.908318 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:05.908344 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:05.908346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:05.908372 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:05.908438 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:05.909083 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.909803 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.910514 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.910812 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.910865 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:05.913158 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:05.913167 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:05.913168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:05.913182 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:05.914618 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.915022 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.915026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.915026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.915087 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.915285 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.915286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.915287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.915424 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:05.915590 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:05.915675 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 21:17:05.915762 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24462](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:05.923873 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:05.917172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37490 (local address 127.20.115.254:45031)
0924 21:17:05.917240 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:05.917242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:05.917253 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:05.917328 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:05.917329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:05.917338 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:05.917394 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:05.917844 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.918325 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.919020 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:05.919224 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:05.919306 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:05.921337 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:05.921339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:05.921339 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:05.921347 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:05.922734 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.923066 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.923067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.923067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.923126 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:05.923325 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:05.923326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:05.923327 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:05.923521 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:05.923604 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:05.923744 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 21:17:05.923811 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260924 21:17:05.924192 24507 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37490
I20260924 21:17:05.924569 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 24569
I20260924 21:17:05.929876 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 24703
I20260924 21:17:05.934439 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 24837
I20260924 21:17:05.938522 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 24475
2026-09-24T21:17:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1881 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24978](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24978](info): set up 2 sockets
Sep 24 21:17:05 dist-test-slave-cxpd krb5kdc[24978](info): commencing operation
krb5kdc: starting...
W20260924 21:17:06.968595 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:07 dist-test-slave-cxpd krb5kdc[24978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:17:07.085590 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43285
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:43285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:07.164610 24991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:07.164830 24991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:07.164855 24991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:07.166054 24991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:07.166098 24991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:07.166113 24991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:07.166126 24991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:07.166138 24991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:07.167647 24991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:43285
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:07.167894 24991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:07.168143 24991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:07.172888 24999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:07.172887 24997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:07.173091 24996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:07.173115 24991 server_base.cc:1034] running on GCE node
I20260924 21:17:07.173408 24991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:07.173691 24991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:07.174880 24991 hybrid_clock.cc:648] HybridClock initialized: now 1790284627174867 us; error 31 us; skew 500 ppm
Sep 24 21:17:07 dist-test-slave-cxpd krb5kdc[24978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284627, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:07.177423 24991 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:17:07.177791 24991 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:07.177862 24991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:07.178011 24991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:07.179078 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "5cdca1a880de47659fc43bb727755c0c"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "0e1eb140ec57055513f903d68f8add62"
server_key_iv: "7833f7fc4e3fcb59886e50c0a800f253"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.179450 24991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "5cdca1a880de47659fc43bb727755c0c"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "0e1eb140ec57055513f903d68f8add62"
server_key_iv: "7833f7fc4e3fcb59886e50c0a800f253"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.180723 24991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:07.181506 25005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.181684 24991 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:07.181782 24991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "5cdca1a880de47659fc43bb727755c0c"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "0e1eb140ec57055513f903d68f8add62"
server_key_iv: "7833f7fc4e3fcb59886e50c0a800f253"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.181866 24991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:07.198505 24991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:07.200860 24991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:07.201027 24991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:07.204860 24991 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:43285
I20260924 21:17:07.204902 25057 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:43285 every 8 connection(s)
I20260924 21:17:07.205207 24991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:07.206087 25058 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:07.208338 25058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c: Bootstrap starting.
I20260924 21:17:07.208983 25058 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:07.209276 25058 log.cc:824] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:07.209971 25058 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c: No bootstrap required, opened a new log
I20260924 21:17:07.210100 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 24991
I20260924 21:17:07.210217 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:07.210388 20943 external_mini_cluster.cc:1428] Setting key 24349b6ac67d2f7f39d329fca5a0f748
I20260924 21:17:07.212570 25058 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdca1a880de47659fc43bb727755c0c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43285 } }
I20260924 21:17:07.212741 25058 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:07.212803 25058 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cdca1a880de47659fc43bb727755c0c, State: Initialized, Role: FOLLOWER
I20260924 21:17:07.212956 25058 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdca1a880de47659fc43bb727755c0c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43285 } }
I20260924 21:17:07.213037 25058 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:07.213060 25058 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:07.213127 25058 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:07.213725 25058 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdca1a880de47659fc43bb727755c0c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43285 } }
I20260924 21:17:07.213829 25058 leader_election.cc:302] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cdca1a880de47659fc43bb727755c0c; no voters:
I20260924 21:17:07.213975 25058 leader_election.cc:288] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:07.214058 25063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:07.214205 25063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [term 1 LEADER]: Becoming Leader. State: Replica: 5cdca1a880de47659fc43bb727755c0c, State: Running, Role: LEADER
I20260924 21:17:07.214213 25058 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:07.214318 25063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdca1a880de47659fc43bb727755c0c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43285 } }
I20260924 21:17:07.215296 25065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cdca1a880de47659fc43bb727755c0c. Latest consensus state: current_term: 1 leader_uuid: "5cdca1a880de47659fc43bb727755c0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdca1a880de47659fc43bb727755c0c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43285 } } }
I20260924 21:17:07.215375 25065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:07.215502 25064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cdca1a880de47659fc43bb727755c0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cdca1a880de47659fc43bb727755c0c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43285 } } }
I20260924 21:17:07.215561 25064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:07.215610 25075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:07.216095 25075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:07.217669 25075 catalog_manager.cc:1347] Generated new cluster ID: 85737b23f7914b48b377d2b44088cb16
I20260924 21:17:07.217736 25075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:17:07 dist-test-slave-cxpd krb5kdc[24978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:07.229514 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.210950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50328 (local address 127.20.115.254:43285)
0924 21:17:07.211125 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.211135 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.211168 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:17:07.212520 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.212529 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.212591 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.212741 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:07.213473 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.214857 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.216012 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.216478 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.216600 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.225977 (+ 9377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:07.225981 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:07.225981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:07.225994 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:07.227875 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.228277 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.228279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.228280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.228386 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.228589 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.228590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.228590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.228895 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:07.229037 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.229157 (+ 120us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.229282 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260924 21:17:07.232542 25075 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:07.233243 25075 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:07.239871 25075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c: Generated new TSK 0
I20260924 21:17:07.240027 25075 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:17:07.269294 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43285
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:07.350283 25084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:07.350478 25084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:07.350502 25084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:07.351552 25084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:07.351583 25084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:07.351608 25084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:07.353070 25084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:07.353247 25084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:07.353410 25084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:17:07.358294 25084 server_base.cc:1034] running on GCE node
W20260924 21:17:07.358263 25092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:07.358263 25090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:07.358266 25089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:07.358728 25084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:07.359000 25084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:07.360145 25084 hybrid_clock.cc:648] HybridClock initialized: now 1790284627360132 us; error 26 us; skew 500 ppm
Sep 24 21:17:07 dist-test-slave-cxpd krb5kdc[24978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284627, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:07.362776 25084 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:17:07.363166 25084 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:07.363217 25084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:07.363308 25084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:07.364312 25084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "81475e983d124d27a1a093bb4e259f3a"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "ffec4b91cf7986f4bc2dcd2423f6aacd"
server_key_iv: "3eaa7bee342446fa0fb04baa1de63d66"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.364607 25084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "81475e983d124d27a1a093bb4e259f3a"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "ffec4b91cf7986f4bc2dcd2423f6aacd"
server_key_iv: "3eaa7bee342446fa0fb04baa1de63d66"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.365906 25084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 21:17:07.366590 25098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.366763 25084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:07.366864 25084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "81475e983d124d27a1a093bb4e259f3a"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "ffec4b91cf7986f4bc2dcd2423f6aacd"
server_key_iv: "3eaa7bee342446fa0fb04baa1de63d66"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.366956 25084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:07.381052 25084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:07.383260 25084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:07.383419 25084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:07.383641 25084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:07.383937 25084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:07.383988 25084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.384029 25084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:07.384055 25084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.389870 25084 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:35237
I20260924 21:17:07.389904 25211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:35237 every 8 connection(s)
I20260924 21:17:07.390172 25084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:17:07.393735 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 25084
I20260924 21:17:07.393965 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:07.394147 20943 external_mini_cluster.cc:1428] Setting key d5c661bbe553acde9607e70e09dc80e7
Sep 24 21:17:07 dist-test-slave-cxpd krb5kdc[24978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284627, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:07.400244 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.390918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:39381 (local address 127.20.115.254:43285)
0924 21:17:07.390997 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.390999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.391172 (+ 173us) server_negotiation.cc:407] Connection header received
0924 21:17:07.391677 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.391678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.391690 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.391749 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:07.392683 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.393287 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.394418 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.394709 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.394774 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.397078 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:07.397080 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:07.397080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:07.397088 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:07.398723 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.399249 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.399250 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.399250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.399304 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.399594 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.399594 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.399594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.399697 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:07.399746 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.400108 (+ 362us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.400181 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 21:17:07.400497 25214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.390907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43285 (local address 127.20.115.193:39381)
0924 21:17:07.391059 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:07.391093 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:07.391575 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:07.391818 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:07.391821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:07.392077 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:07.392584 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.392588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.393383 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.393385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.394285 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.394288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.394799 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.394800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.394987 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.395555 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:07.395564 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:07.396995 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:07.398809 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:07.398816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:07.398819 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:07.399162 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:07.399374 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:07.399375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:07.399376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:07.399523 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:07.399765 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:07.399771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:07.399946 (+ 175us) client_negotiation.cc:769] Sending connection context
0924 21:17:07.400085 (+ 139us) client_negotiation.cc:240] Negotiation successful
0924 21:17:07.400251 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":79,"threads_started":1}
I20260924 21:17:07.401129 25212 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43285
I20260924 21:17:07.401238 25212 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:07.401391 25212 heartbeater.cc:511] Master 127.20.115.254:43285 requested a full tablet report, sending...
I20260924 21:17:07.401970 25022 ts_manager.cc:194] Registered new tserver with Master: 81475e983d124d27a1a093bb4e259f3a (127.20.115.193:35237)
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
I20260924 21:17:07.402657 25022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:39381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:17:07.419833 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43285
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:07.504163 25217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:07.504451 25217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:07.504491 25217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:07.506117 25217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:07.506181 25217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:07.506227 25217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:07.507956 25217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:07.508219 25217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:07.508525 25217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:07.514703 25225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:07.514730 25223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:07.514730 25222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:07.515789 25217 server_base.cc:1034] running on GCE node
I20260924 21:17:07.515986 25217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:07.516253 25217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:07.517505 25217 hybrid_clock.cc:648] HybridClock initialized: now 1790284627517484 us; error 33 us; skew 500 ppm
Sep 24 21:17:07 dist-test-slave-cxpd krb5kdc[24978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284627, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:07.520853 25217 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:17:07.521270 25217 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:07.521343 25217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:07.521510 25217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:07.523046 25217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "d7e49ad7583449758099fa03af489f96"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "b347c9fa1d506f28fc4487dcdf36f36e"
server_key_iv: "67c85c3669a376f434fb3cc304a514d3"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.523505 25217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "d7e49ad7583449758099fa03af489f96"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "b347c9fa1d506f28fc4487dcdf36f36e"
server_key_iv: "67c85c3669a376f434fb3cc304a514d3"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.525342 25217 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260924 21:17:07.526327 25231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.526527 25217 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:07.526629 25217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "d7e49ad7583449758099fa03af489f96"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "b347c9fa1d506f28fc4487dcdf36f36e"
server_key_iv: "67c85c3669a376f434fb3cc304a514d3"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.526715 25217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:07.565680 25217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:07.567699 25217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:07.567857 25217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:07.568106 25217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:07.568426 25217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:07.568459 25217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.568482 25217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:07.568529 25217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.574545 25217 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:37997
I20260924 21:17:07.574589 25344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:37997 every 8 connection(s)
I20260924 21:17:07.574856 25217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:17:07 dist-test-slave-cxpd krb5kdc[24978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284627, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:07.583550 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.575553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:47861 (local address 127.20.115.254:43285)
0924 21:17:07.575609 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.575611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.575806 (+ 195us) server_negotiation.cc:407] Connection header received
0924 21:17:07.576328 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.576329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.576341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.576410 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:07.577332 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.577840 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.578823 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.579030 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.579106 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.581140 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:07.581141 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:07.581141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:07.581148 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:07.582322 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.582702 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.582703 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.582703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.582751 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.583031 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.583032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.583032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.583127 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:07.583177 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.583427 (+ 250us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.583485 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:17:07.583734 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.575530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43285 (local address 127.20.115.194:47861)
0924 21:17:07.575683 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:07.575717 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:07.576224 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:07.576493 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:07.576496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:07.576747 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:07.577230 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.577235 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.577932 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.577933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.578723 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.578725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.579108 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.579109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.579230 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.579660 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:07.579668 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:07.581057 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:07.582416 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:07.582422 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:07.582423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:07.582635 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:07.582831 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:07.582833 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:07.582833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:07.582945 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:07.583190 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:07.583195 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:07.583308 (+ 113us) client_negotiation.cc:769] Sending connection context
0924 21:17:07.583403 (+ 95us) client_negotiation.cc:240] Negotiation successful
0924 21:17:07.583520 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":93,"threads_started":1}
I20260924 21:17:07.584198 25345 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43285
I20260924 21:17:07.584302 25345 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:07.584438 25345 heartbeater.cc:511] Master 127.20.115.254:43285 requested a full tablet report, sending...
I20260924 21:17:07.584700 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 25217
I20260924 21:17:07.584897 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:07.584897 25022 ts_manager.cc:194] Registered new tserver with Master: d7e49ad7583449758099fa03af489f96 (127.20.115.194:37997)
I20260924 21:17:07.585121 20943 external_mini_cluster.cc:1428] Setting key 996de3d0377a4502d66eadf6f51cd944
I20260924 21:17:07.585376 25022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:47861
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:17:07.607816 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43285
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:07.685959 25350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:07.686168 25350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:07.686205 25350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:07.687323 25350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:07.687382 25350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:07.687426 25350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:07.688927 25350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:07.689186 25350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:07.689404 25350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:07.694169 25356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:07.694192 25355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:07.694222 25350 server_base.cc:1034] running on GCE node
W20260924 21:17:07.694170 25358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:07.694545 25350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:07.694767 25350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:07.695931 25350 hybrid_clock.cc:648] HybridClock initialized: now 1790284627695923 us; error 55 us; skew 500 ppm
Sep 24 21:17:07 dist-test-slave-cxpd krb5kdc[24978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284627, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:07.698381 25350 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:17:07.698699 25350 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:07.698769 25350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:07.698913 25350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:07.699875 25350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "246eea4d7bed476bb17b0dcf40b8a6a2"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "615c410ed7ce5048f717fcdd70f4e90c"
server_key_iv: "2be0308860395817221cc99acc04feaa"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.700225 25350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "246eea4d7bed476bb17b0dcf40b8a6a2"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "615c410ed7ce5048f717fcdd70f4e90c"
server_key_iv: "2be0308860395817221cc99acc04feaa"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.701617 25350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:07.702405 25364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.702593 25350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:07.702687 25350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "246eea4d7bed476bb17b0dcf40b8a6a2"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "615c410ed7ce5048f717fcdd70f4e90c"
server_key_iv: "2be0308860395817221cc99acc04feaa"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.702770 25350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:07.747795 25350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:07.749851 25350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:07.750017 25350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:07.750252 25350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:07.750586 25350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:07.750648 25350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.750686 25350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:07.750715 25350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.756546 25350 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:43587
I20260924 21:17:07.756578 25477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:43587 every 8 connection(s)
I20260924 21:17:07.756896 25350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:17:07.762426 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 25350
I20260924 21:17:07.762557 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
Sep 24 21:17:07 dist-test-slave-cxpd krb5kdc[24978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284627, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:07.762760 20943 external_mini_cluster.cc:1428] Setting key 4b766b24fde47a62dd3dd6f75adec326
I20260924 21:17:07.765748 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.757620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:48691 (local address 127.20.115.254:43285)
0924 21:17:07.757696 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.757698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.757893 (+ 195us) server_negotiation.cc:407] Connection header received
0924 21:17:07.758386 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.758391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.758402 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.758456 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:07.759351 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.759838 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.760831 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.761038 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.761097 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.763158 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:07.763160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:07.763160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:07.763174 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:07.764602 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.765012 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.765013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.765013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.765066 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.765262 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.765263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.765263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.765354 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:07.765400 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.765622 (+ 222us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.765685 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:17:07.765934 25480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.757569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43285 (local address 127.20.115.195:48691)
0924 21:17:07.757743 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:07.757779 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:07.758282 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:07.758532 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:07.758535 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:07.758773 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:07.759260 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.759265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.759915 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.759916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.760702 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.760704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.761113 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.761113 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.761211 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.761621 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:07.761629 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:07.763058 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:07.764675 (+ 1617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:07.764680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:07.764682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:07.764960 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:07.765128 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:07.765128 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:07.765128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:07.765219 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:07.765404 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:07.765408 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:07.765511 (+ 103us) client_negotiation.cc:769] Sending connection context
0924 21:17:07.765594 (+ 83us) client_negotiation.cc:240] Negotiation successful
0924 21:17:07.765709 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":96,"threads_started":1}
I20260924 21:17:07.766469 25478 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43285
I20260924 21:17:07.766566 25478 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:07.766681 25478 heartbeater.cc:511] Master 127.20.115.254:43285 requested a full tablet report, sending...
I20260924 21:17:07.767009 25021 ts_manager.cc:194] Registered new tserver with Master: 246eea4d7bed476bb17b0dcf40b8a6a2 (127.20.115.195:43587)
I20260924 21:17:07.767401 25021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:48691
I20260924 21:17:07.774605 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:17:07.781618 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.775907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50334 (local address 127.20.115.254:43285)
0924 21:17:07.775966 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.775969 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.776079 (+ 110us) server_negotiation.cc:407] Connection header received
0924 21:17:07.776113 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.776114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.776122 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.776173 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:07.776688 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.777221 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.777912 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.778098 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.778163 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.779321 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:07.779323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:07.779323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:07.779330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:07.780604 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.780985 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.780986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.780986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.781042 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:07.781197 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:07.781197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:07.781198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:07.781329 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:07.781398 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.781495 (+ 97us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.781557 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 21:17:07.783255 25021 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:17:07.784986 25021 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:07.791702 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.787483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:35237 (local address 127.0.0.1:46950)
0924 21:17:07.787956 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:07.787965 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:07.788019 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:07.788196 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:07.788199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:07.788202 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:07.788268 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.788270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.789643 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.789644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.790587 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.790589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.791455 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.791456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.791519 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.791530 (+ 11us) client_negotiation.cc:769] Sending connection context
0924 21:17:07.791586 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 21:17:07.791622 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":85,"threads_started":1}
I20260924 21:17:07.791993 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.787722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46950 (local address 127.20.115.193:35237)
0924 21:17:07.787915 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.787920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.788043 (+ 123us) server_negotiation.cc:407] Connection header received
0924 21:17:07.788088 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.788090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.788104 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.788183 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:07.788366 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.789526 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.790673 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.791372 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.791454 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.791507 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.791622 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.791665 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":92,"threads_started":1}
I20260924 21:17:07.792058 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.787467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:43587 (local address 127.0.0.1:57288)
0924 21:17:07.787888 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:07.787899 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:07.787983 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:07.788416 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:07.788417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:07.788419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:07.788483 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.788485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.789443 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.789444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.790811 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.790813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.791725 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.791726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.791888 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.791892 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:07.791957 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 21:17:07.791995 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":61,"thread_start_us":101,"threads_started":1}
I20260924 21:17:07.792130 25492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.787557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57288 (local address 127.20.115.195:43587)
0924 21:17:07.788178 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.788182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.788193 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:07.788231 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.788234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.788250 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.788325 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:07.788607 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.789332 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.790900 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.791639 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.791723 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.791773 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.791996 (+ 223us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.792039 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":75,"threads_started":1}
I20260924 21:17:07.792928 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.787465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:37997 (local address 127.0.0.1:43848)
0924 21:17:07.787647 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:07.787690 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:07.787760 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:07.788268 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:07.788269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:07.788270 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:07.788313 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.788314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.789626 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.789627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.790810 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.790813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.792672 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.792673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.792735 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.792739 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:07.792815 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:17:07.792859 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":28,"thread_start_us":70,"threads_started":1}
I20260924 21:17:07.792963 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.787724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43848 (local address 127.20.115.194:37997)
0924 21:17:07.787903 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.787907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.787915 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:07.787949 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.787951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.787963 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.788054 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:07.788395 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.789526 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.791813 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.792589 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.792667 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.792719 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.792843 (+ 124us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.792889 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":83,"threads_started":1}
I20260924 21:17:07.792865 25412 tablet_service.cc:1467] Processing CreateTablet for tablet 3e38883b5aa24c0cbda31065118c6aec (DEFAULT_TABLE table=test-table [id=40c06bff52f34045b2bf1055d7794c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:07.793218 25412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e38883b5aa24c0cbda31065118c6aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:07.793165 25146 tablet_service.cc:1467] Processing CreateTablet for tablet 3e38883b5aa24c0cbda31065118c6aec (DEFAULT_TABLE table=test-table [id=40c06bff52f34045b2bf1055d7794c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:07.793404 25146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e38883b5aa24c0cbda31065118c6aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:07.795665 25495 tablet_bootstrap.cc:492] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: Bootstrap starting.
I20260924 21:17:07.795799 25496 tablet_bootstrap.cc:492] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a: Bootstrap starting.
I20260924 21:17:07.796357 25495 tablet_bootstrap.cc:654] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:07.796397 25496 tablet_bootstrap.cc:654] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:07.796437 25279 tablet_service.cc:1467] Processing CreateTablet for tablet 3e38883b5aa24c0cbda31065118c6aec (DEFAULT_TABLE table=test-table [id=40c06bff52f34045b2bf1055d7794c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:07.796804 25279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e38883b5aa24c0cbda31065118c6aec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:07.797003 25495 log.cc:824] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:07.797004 25496 log.cc:824] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:07.797699 25495 tablet_bootstrap.cc:492] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: No bootstrap required, opened a new log
I20260924 21:17:07.797717 25496 tablet_bootstrap.cc:492] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a: No bootstrap required, opened a new log
I20260924 21:17:07.797781 25496 ts_tablet_manager.cc:1397] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:17:07.797783 25495 ts_tablet_manager.cc:1397] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:17:07.799460 25495 raft_consensus.cc:348] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } }
I20260924 21:17:07.799613 25495 raft_consensus.cc:374] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:07.799656 25495 raft_consensus.cc:729] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246eea4d7bed476bb17b0dcf40b8a6a2, State: Initialized, Role: FOLLOWER
I20260924 21:17:07.799783 25495 consensus_queue.cc:260] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } }
I20260924 21:17:07.799762 25496 raft_consensus.cc:348] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } }
I20260924 21:17:07.799926 25496 raft_consensus.cc:374] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:07.799966 25496 raft_consensus.cc:729] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81475e983d124d27a1a093bb4e259f3a, State: Initialized, Role: FOLLOWER
I20260924 21:17:07.800081 25495 ts_tablet_manager.cc:1428] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:17:07.800077 25496 consensus_queue.cc:260] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } }
I20260924 21:17:07.800220 25478 heartbeater.cc:503] Master 127.20.115.254:43285 was elected leader, sending a full tablet report...
I20260924 21:17:07.800339 25496 ts_tablet_manager.cc:1428] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 21:17:07.800386 25212 heartbeater.cc:503] Master 127.20.115.254:43285 was elected leader, sending a full tablet report...
I20260924 21:17:07.800581 25499 tablet_bootstrap.cc:492] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: Bootstrap starting.
I20260924 21:17:07.801177 25499 tablet_bootstrap.cc:654] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:07.801661 25499 log.cc:824] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:07.802464 25499 tablet_bootstrap.cc:492] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: No bootstrap required, opened a new log
I20260924 21:17:07.802542 25499 ts_tablet_manager.cc:1397] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:07.803810 25499 raft_consensus.cc:348] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } }
I20260924 21:17:07.803949 25499 raft_consensus.cc:374] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:07.803987 25499 raft_consensus.cc:729] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e49ad7583449758099fa03af489f96, State: Initialized, Role: FOLLOWER
I20260924 21:17:07.804097 25499 consensus_queue.cc:260] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } }
I20260924 21:17:07.804346 25499 ts_tablet_manager.cc:1428] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:17:07.804414 25345 heartbeater.cc:503] Master 127.20.115.254:43285 was elected leader, sending a full tablet report...
I20260924 21:17:07.807562 25503 raft_consensus.cc:482] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:07.807628 25503 raft_consensus.cc:504] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } }
I20260924 21:17:07.807855 25503 leader_election.cc:288] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 81475e983d124d27a1a093bb4e259f3a (127.20.115.193:35237), 246eea4d7bed476bb17b0dcf40b8a6a2 (127.20.115.195:43587)
I20260924 21:17:07.810526 25492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.808088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:59231 (local address 127.20.115.195:43587)
0924 21:17:07.808143 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.808145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.808152 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:17:07.808175 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.808176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.808184 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.808241 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:07.808388 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.808908 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.809799 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.810173 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.810264 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.810272 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.810428 (+ 156us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.810471 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:17:07.810526 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.807989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:43587 (local address 127.20.115.194:59231)
0924 21:17:07.808029 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:07.808041 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:07.808087 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:07.808244 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:07.808245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:07.808246 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:07.808316 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.808317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.808986 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.808987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.809697 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.809699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.810263 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.810264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.810344 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.810347 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:07.810421 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 21:17:07.810459 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260924 21:17:07.810782 25504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.807999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:35237 (local address 127.20.115.194:34903)
0924 21:17:07.808253 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:07.808262 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:07.808321 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:07.808466 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:07.808467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:07.808469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:07.808538 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.808540 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.809222 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.809223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.809951 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:07.809952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.810500 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.810501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:07.810576 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.810579 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:07.810668 (+ 89us) client_negotiation.cc:240] Negotiation successful
0924 21:17:07.810711 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":122,"thread_start_us":93,"threads_started":1}
I20260924 21:17:07.810833 25494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.808065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:34903 (local address 127.20.115.193:35237)
0924 21:17:07.808108 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.808110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.808327 (+ 217us) server_negotiation.cc:407] Connection header received
0924 21:17:07.808384 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.808385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.808409 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.808463 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:07.808613 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.809143 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.810049 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.810416 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.810500 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.810507 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.810730 (+ 223us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.810776 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:17:07.810845 25432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e38883b5aa24c0cbda31065118c6aec" candidate_uuid: "d7e49ad7583449758099fa03af489f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" is_pre_election: true
I20260924 21:17:07.810974 25432 raft_consensus.cc:2454] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e49ad7583449758099fa03af489f96 in term 0.
I20260924 21:17:07.811185 25232 leader_election.cc:302] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246eea4d7bed476bb17b0dcf40b8a6a2, d7e49ad7583449758099fa03af489f96; no voters:
I20260924 21:17:07.811331 25503 raft_consensus.cc:2781] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:07.811319 25166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e38883b5aa24c0cbda31065118c6aec" candidate_uuid: "d7e49ad7583449758099fa03af489f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81475e983d124d27a1a093bb4e259f3a" is_pre_election: true
I20260924 21:17:07.811378 25503 raft_consensus.cc:482] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:07.811405 25503 raft_consensus.cc:3037] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:07.811451 25166 raft_consensus.cc:2454] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d7e49ad7583449758099fa03af489f96 in term 0.
I20260924 21:17:07.812028 25503 raft_consensus.cc:504] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } }
I20260924 21:17:07.812168 25503 leader_election.cc:288] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [CANDIDATE]: Term 1 election: Requested vote from peers 81475e983d124d27a1a093bb4e259f3a (127.20.115.193:35237), 246eea4d7bed476bb17b0dcf40b8a6a2 (127.20.115.195:43587)
I20260924 21:17:07.812320 25432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e38883b5aa24c0cbda31065118c6aec" candidate_uuid: "d7e49ad7583449758099fa03af489f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2"
I20260924 21:17:07.812331 25166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e38883b5aa24c0cbda31065118c6aec" candidate_uuid: "d7e49ad7583449758099fa03af489f96" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "81475e983d124d27a1a093bb4e259f3a"
I20260924 21:17:07.812388 25432 raft_consensus.cc:3037] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:07.812399 25166 raft_consensus.cc:3037] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:07.813126 25432 raft_consensus.cc:2454] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e49ad7583449758099fa03af489f96 in term 1.
I20260924 21:17:07.813134 25166 raft_consensus.cc:2454] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d7e49ad7583449758099fa03af489f96 in term 1.
I20260924 21:17:07.813282 25232 leader_election.cc:302] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246eea4d7bed476bb17b0dcf40b8a6a2, d7e49ad7583449758099fa03af489f96; no voters:
I20260924 21:17:07.813387 25503 raft_consensus.cc:2781] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:07.813555 25503 raft_consensus.cc:686] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 1 LEADER]: Becoming Leader. State: Replica: d7e49ad7583449758099fa03af489f96, State: Running, Role: LEADER
I20260924 21:17:07.813647 25503 consensus_queue.cc:237] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } }
I20260924 21:17:07.814456 25021 catalog_manager.cc:5495] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 reported cstate change: term changed from 0 to 1, leader changed from <none> to d7e49ad7583449758099fa03af489f96 (127.20.115.194). New cstate: current_term: 1 leader_uuid: "d7e49ad7583449758099fa03af489f96" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7e49ad7583449758099fa03af489f96" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 37997 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 } health_report { overall_health: UNKNOWN } } }
I20260924 21:17:07.821560 25493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.819229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.20.115.194:37997)
0924 21:17:07.819286 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.819287 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.819295 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:07.819324 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.819325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.819347 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.819408 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:07.819520 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.819982 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.820707 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:07.821030 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:07.821096 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:07.821243 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:07.821331 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:07.821392 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.821429 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.821445 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:17:07.823175 25432 raft_consensus.cc:1261] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [term 1 FOLLOWER]: Refusing update from remote peer d7e49ad7583449758099fa03af489f96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:07.823179 25166 raft_consensus.cc:1261] T 3e38883b5aa24c0cbda31065118c6aec P 81475e983d124d27a1a093bb4e259f3a [term 1 FOLLOWER]: Refusing update from remote peer d7e49ad7583449758099fa03af489f96: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:07.823472 25505 consensus_queue.cc:1035] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "246eea4d7bed476bb17b0dcf40b8a6a2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43587 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:07.823637 25503 consensus_queue.cc:1035] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [LEADER]: Connected to new peer: Peer: permanent_uuid: "81475e983d124d27a1a093bb4e259f3a" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 21:17:07.825304 25346 tablet.cc:2369] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:17:07.825978 25508 mvcc.cc:205] Tried to move back new op lower bound from 7333005835561299968 to 7333005835525373952. Current Snapshot: MvccSnapshot[applied={T|T < 7333005835561299968}]
I20260924 21:17:07.826102 25510 mvcc.cc:205] Tried to move back new op lower bound from 7333005835561299968 to 7333005835525373952. Current Snapshot: MvccSnapshot[applied={T|T < 7333005835561299968}]
I20260924 21:17:07.826571 25509 mvcc.cc:205] Tried to move back new op lower bound from 7333005835561299968 to 7333005835525373952. Current Snapshot: MvccSnapshot[applied={T|T < 7333005835561299968}]
I20260924 21:17:07.828024 25021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:07.828120 25021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:07.828811 25021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5cdca1a880de47659fc43bb727755c0c: Sending DeleteTablet for 3 replicas of tablet 3e38883b5aa24c0cbda31065118c6aec
I20260924 21:17:07.829140 25279 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e38883b5aa24c0cbda31065118c6aec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:07 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:43848
I20260924 21:17:07.829150 25146 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e38883b5aa24c0cbda31065118c6aec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:07 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:46950
I20260924 21:17:07.829145 25412 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e38883b5aa24c0cbda31065118c6aec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:07 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:57288
I20260924 21:17:07.829337 25518 tablet_replica.cc:330] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: stopping tablet replica
I20260924 21:17:07.829416 25518 raft_consensus.cc:2229] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:07.829542 25519 tablet_replica.cc:330] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: stopping tablet replica
I20260924 21:17:07.829612 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 25084
I20260924 21:17:07.830411 25519 raft_consensus.cc:2229] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:17:07.830569 25519 raft_consensus.cc:2258] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:07.830999 25519 ts_tablet_manager.cc:1905] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:07.831136 25518 raft_consensus.cc:2258] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:07.831547 25518 ts_tablet_manager.cc:1905] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:07.832687 25519 ts_tablet_manager.cc:1918] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:07.832770 25519 log.cc:1196] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/3e38883b5aa24c0cbda31065118c6aec
I20260924 21:17:07.832954 25518 ts_tablet_manager.cc:1918] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:07.833019 25518 log.cc:1196] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/3e38883b5aa24c0cbda31065118c6aec
I20260924 21:17:07.833074 25519 ts_tablet_manager.cc:1939] T 3e38883b5aa24c0cbda31065118c6aec P d7e49ad7583449758099fa03af489f96: Deleting consensus metadata
I20260924 21:17:07.833283 25518 ts_tablet_manager.cc:1939] T 3e38883b5aa24c0cbda31065118c6aec P 246eea4d7bed476bb17b0dcf40b8a6a2: Deleting consensus metadata
I20260924 21:17:07.833428 25007 catalog_manager.cc:4841] TS d7e49ad7583449758099fa03af489f96 (127.20.115.194:37997): tablet 3e38883b5aa24c0cbda31065118c6aec (table test-table [id=40c06bff52f34045b2bf1055d7794c6c]) successfully deleted
I20260924 21:17:07.833925 25006 catalog_manager.cc:4841] TS 246eea4d7bed476bb17b0dcf40b8a6a2 (127.20.115.195:43587): tablet 3e38883b5aa24c0cbda31065118c6aec (table test-table [id=40c06bff52f34045b2bf1055d7794c6c]) successfully deleted
W20260924 21:17:07.835507 25009 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:35237 (error 108)
I20260924 21:17:07.835706 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 25217
I20260924 21:17:07.836647 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.836521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:35237 (local address 127.0.0.1:46962)
0924 21:17:07.836583 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:07.836620 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260924 21:17:07.836766 25009 catalog_manager.cc:4584] TS 81475e983d124d27a1a093bb4e259f3a (127.20.115.193:35237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e38883b5aa24c0cbda31065118c6aec: Network error: Client connection negotiation failed: client connection to 127.20.115.193:35237: connect: Connection refused (error 111)
I20260924 21:17:07.841166 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 25350
I20260924 21:17:07.845870 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 24991
2026-09-24T21:17:07Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1907 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-24T21:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:07Z Disabled control of system clock
I20260924 21:17:07.873178 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45701
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:44613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:45701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 21:17:07.951301 25525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:07.951488 25525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:07.951509 25525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:07.952543 25525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:07.952572 25525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:07.952585 25525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:07.952597 25525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:07.952610 25525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:07.953925 25525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:44613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:45701
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:07.954115 25525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:07.954337 25525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:07.959170 25530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:07.959206 25533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:07.959225 25525 server_base.cc:1034] running on GCE node
W20260924 21:17:07.959177 25531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:07.959659 25525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:07.959930 25525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:07.961071 25525 hybrid_clock.cc:648] HybridClock initialized: now 1790284627961049 us; error 36 us; skew 500 ppm
I20260924 21:17:07.962309 25525 webserver.cc:460] Webserver started at http://127.20.115.254:45959/ using document root <none> and password file <none>
I20260924 21:17:07.962558 25525 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:07.962603 25525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:07.962757 25525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:07.963776 25525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "af16c765a1d34a838207a0a9e03846b6"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "140d413ec7ec4c416681a5ba125e360b"
server_key_iv: "6a2b619d5c0ae16b9adf096f51529326"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.964130 25525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "af16c765a1d34a838207a0a9e03846b6"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "140d413ec7ec4c416681a5ba125e360b"
server_key_iv: "6a2b619d5c0ae16b9adf096f51529326"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.965461 25525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:17:07.966163 25539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:07.966357 25525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:07.966452 25525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "af16c765a1d34a838207a0a9e03846b6"
format_stamp: "Formatted at 2026-09-24 21:17:07 on dist-test-slave-cxpd"
server_key: "140d413ec7ec4c416681a5ba125e360b"
server_key_iv: "6a2b619d5c0ae16b9adf096f51529326"
server_key_version: "encryptionkey@0"
I20260924 21:17:07.966545 25525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:07.969712 25525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:07.969959 25525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:07.970091 25525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:07.973815 25525 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:45701
I20260924 21:17:07.973865 25591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:45701 every 8 connection(s)
I20260924 21:17:07.974175 25525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:07.974766 25592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:07.977005 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 25525
I20260924 21:17:07.977115 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:07.977296 20943 external_mini_cluster.cc:1428] Setting key 3e276b14edc6666b4cab8f9038741c21
I20260924 21:17:07.977691 25592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6: Bootstrap starting.
I20260924 21:17:07.978292 25592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:07.978595 25592 log.cc:824] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:07.979354 25592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6: No bootstrap required, opened a new log
I20260924 21:17:07.979830 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:07.977835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.20.115.254:45701)
0924 21:17:07.978041 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:07.978047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:07.978071 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:17:07.978513 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:07.978520 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:07.978567 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:07.978724 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:07.979381 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:07.979384 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:07.979385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:07.979389 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:07.979434 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:07.979449 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:07.979541 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:07.979576 (+ 35us) server_negotiation.cc:299] Negotiation successful
0924 21:17:07.979637 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260924 21:17:07.981050 25592 raft_consensus.cc:348] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af16c765a1d34a838207a0a9e03846b6" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45701 } }
I20260924 21:17:07.981194 25592 raft_consensus.cc:374] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:07.981240 25592 raft_consensus.cc:729] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af16c765a1d34a838207a0a9e03846b6, State: Initialized, Role: FOLLOWER
I20260924 21:17:07.981352 25592 consensus_queue.cc:260] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af16c765a1d34a838207a0a9e03846b6" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45701 } }
I20260924 21:17:07.981706 25592 raft_consensus.cc:388] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:07.981753 25592 raft_consensus.cc:482] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:07.981825 25592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:07.982338 25592 raft_consensus.cc:504] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af16c765a1d34a838207a0a9e03846b6" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45701 } }
I20260924 21:17:07.982453 25592 leader_election.cc:302] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af16c765a1d34a838207a0a9e03846b6; no voters:
I20260924 21:17:07.982621 25592 leader_election.cc:288] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:07.982648 25597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:07.982805 25597 raft_consensus.cc:686] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [term 1 LEADER]: Becoming Leader. State: Replica: af16c765a1d34a838207a0a9e03846b6, State: Running, Role: LEADER
I20260924 21:17:07.982872 25592 sys_catalog.cc:564] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:07.982898 25597 consensus_queue.cc:237] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af16c765a1d34a838207a0a9e03846b6" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45701 } }
I20260924 21:17:07.983218 25598 sys_catalog.cc:455] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af16c765a1d34a838207a0a9e03846b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af16c765a1d34a838207a0a9e03846b6" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45701 } } }
I20260924 21:17:07.983419 25598 sys_catalog.cc:458] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:07.983227 25599 sys_catalog.cc:455] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af16c765a1d34a838207a0a9e03846b6. Latest consensus state: current_term: 1 leader_uuid: "af16c765a1d34a838207a0a9e03846b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af16c765a1d34a838207a0a9e03846b6" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45701 } } }
I20260924 21:17:07.983687 25599 sys_catalog.cc:458] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:07.983822 25606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:07.984300 25606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:07.985724 25606 catalog_manager.cc:1347] Generated new cluster ID: 726caedd35ae4ea3b45ae2fa525bd505
I20260924 21:17:07.985782 25606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:07.997243 25606 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:07.997404 25606 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:08.001657 25606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6: Generated new TSK 0
I20260924 21:17:08.001845 25606 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:17:08.004758 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45701
--builtin_ntp_servers=127.20.115.212:44613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 21:17:08.087579 25616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:08.087774 25616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:08.087795 25616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:08.088991 25616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:08.089030 25616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:08.089054 25616 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:08.090406 25616 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:44613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:08.090628 25616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:08.090939 25616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:08.096714 25622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:08.096678 25621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:08.096675 25624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:08.096933 25616 server_base.cc:1034] running on GCE node
I20260924 21:17:08.097124 25616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:08.097366 25616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:08.098524 25616 hybrid_clock.cc:648] HybridClock initialized: now 1790284628098517 us; error 38 us; skew 500 ppm
I20260924 21:17:08.099983 25616 webserver.cc:460] Webserver started at http://127.20.115.193:46431/ using document root <none> and password file <none>
I20260924 21:17:08.100235 25616 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:08.100281 25616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:08.100416 25616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:08.101523 25616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "0816fa32580d4283beb4f0501c8a5437"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "c6fe50c6b3abab93c8492a41db9bacde"
server_key_iv: "0900a4afe0897b122e72a6394550c89c"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.101953 25616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "0816fa32580d4283beb4f0501c8a5437"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "c6fe50c6b3abab93c8492a41db9bacde"
server_key_iv: "0900a4afe0897b122e72a6394550c89c"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.103264 25616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:08.104043 25630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.104240 25616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:08.104336 25616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "0816fa32580d4283beb4f0501c8a5437"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "c6fe50c6b3abab93c8492a41db9bacde"
server_key_iv: "0900a4afe0897b122e72a6394550c89c"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.104418 25616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:08.107684 25616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:08.107944 25616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:08.108085 25616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:08.108326 25616 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:08.108623 25616 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:08.108677 25616 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.108714 25616 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:08.108740 25616 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.115350 25616 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:41947
I20260924 21:17:08.115389 25743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:41947 every 8 connection(s)
I20260924 21:17:08.115691 25616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:17:08.118705 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.116393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:56997 (local address 127.20.115.254:45701)
0924 21:17:08.116448 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.116451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.116649 (+ 198us) server_negotiation.cc:407] Connection header received
0924 21:17:08.117196 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.117197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.117205 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.117267 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.118428 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.118429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.118429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.118433 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.118448 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.118451 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.118531 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.118622 (+ 91us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.118637 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 21:17:08.118891 25746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.116365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45701 (local address 127.20.115.193:56997)
0924 21:17:08.116526 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.116564 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.117086 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.117336 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.117340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.117349 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:08.118269 (+ 920us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:08.118270 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:08.118296 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:08.118322 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:08.118543 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:08.118548 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:08.118551 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:08.118628 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 21:17:08.118687 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260924 21:17:08.119382 25744 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45701
I20260924 21:17:08.119455 25744 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:08.119601 25744 heartbeater.cc:511] Master 127.20.115.254:45701 requested a full tablet report, sending...
I20260924 21:17:08.119730 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 25616
I20260924 21:17:08.119831 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:08.120014 20943 external_mini_cluster.cc:1428] Setting key ecd47aec998181b9e263006bf1b186f4
I20260924 21:17:08.120205 25556 ts_manager.cc:194] Registered new tserver with Master: 0816fa32580d4283beb4f0501c8a5437 (127.20.115.193:41947)
I20260924 21:17:08.121053 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45701
--builtin_ntp_servers=127.20.115.212:44613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 21:17:08.200287 25747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:08.200484 25747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:08.200512 25747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:08.201676 25747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:08.201716 25747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:08.201741 25747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:08.203002 25747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:44613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:08.203189 25747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:08.203418 25747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:08.208300 25755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:08.208284 25752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:08.208276 25753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:08.208530 25747 server_base.cc:1034] running on GCE node
I20260924 21:17:08.208731 25747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:08.208983 25747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:08.210145 25747 hybrid_clock.cc:648] HybridClock initialized: now 1790284628210138 us; error 36 us; skew 500 ppm
I20260924 21:17:08.211318 25747 webserver.cc:460] Webserver started at http://127.20.115.194:45557/ using document root <none> and password file <none>
I20260924 21:17:08.211575 25747 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:08.211642 25747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:08.211779 25747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:08.212852 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "9929f415835048e7a08c7d31d8d2826a"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "b20f817590af43a5e7e130f9d29cdd1e"
server_key_iv: "36774595d63c6395f25ac79923c4d375"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.213229 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "9929f415835048e7a08c7d31d8d2826a"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "b20f817590af43a5e7e130f9d29cdd1e"
server_key_iv: "36774595d63c6395f25ac79923c4d375"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.214537 25747 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:08.215351 25761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.215529 25747 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:08.215621 25747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "9929f415835048e7a08c7d31d8d2826a"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "b20f817590af43a5e7e130f9d29cdd1e"
server_key_iv: "36774595d63c6395f25ac79923c4d375"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.215706 25747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:08.218966 25747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:08.219210 25747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:08.219341 25747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:08.219545 25747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:08.219834 25747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:08.219887 25747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.219924 25747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:08.219954 25747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.225517 25747 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:43003
I20260924 21:17:08.225561 25874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:43003 every 8 connection(s)
I20260924 21:17:08.225854 25747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:17:08.228621 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.226467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:57013 (local address 127.20.115.254:45701)
0924 21:17:08.226508 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.226510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.226770 (+ 260us) server_negotiation.cc:407] Connection header received
0924 21:17:08.227242 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.227243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.227249 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.227307 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.228368 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.228369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.228370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.228375 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.228389 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.228393 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.228444 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.228518 (+ 74us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.228540 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:17:08.228829 25877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.226468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45701 (local address 127.20.115.194:57013)
0924 21:17:08.226660 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.226690 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.227141 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.227372 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.227376 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.227385 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:08.228220 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:08.228221 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:08.228246 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:08.228272 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:08.228444 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:08.228448 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:08.228451 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:08.228519 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 21:17:08.228598 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
I20260924 21:17:08.229447 25875 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45701
I20260924 21:17:08.229554 25875 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:08.229696 25875 heartbeater.cc:511] Master 127.20.115.254:45701 requested a full tablet report, sending...
I20260924 21:17:08.230055 25555 ts_manager.cc:194] Registered new tserver with Master: 9929f415835048e7a08c7d31d8d2826a (127.20.115.194:43003)
I20260924 21:17:08.235291 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 25747
I20260924 21:17:08.235378 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:08.235502 20943 external_mini_cluster.cc:1428] Setting key 9825ab5fba85698fcdcb1ad3f8b6f734
I20260924 21:17:08.236284 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45701
--builtin_ntp_servers=127.20.115.212:44613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260924 21:17:08.314218 25878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:08.314396 25878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:08.314417 25878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:08.315490 25878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:08.315522 25878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:08.315546 25878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:08.316752 25878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:44613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45701
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:08.317003 25878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:08.317271 25878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:08.322256 25886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:08.322237 25883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:08.322256 25884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:08.322286 25878 server_base.cc:1034] running on GCE node
I20260924 21:17:08.322623 25878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:08.322865 25878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:08.324007 25878 hybrid_clock.cc:648] HybridClock initialized: now 1790284628323990 us; error 33 us; skew 500 ppm
I20260924 21:17:08.325289 25878 webserver.cc:460] Webserver started at http://127.20.115.195:46479/ using document root <none> and password file <none>
I20260924 21:17:08.325539 25878 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:08.325585 25878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:08.325754 25878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:08.326810 25878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "870a5d4205904f09a1da573b08858ca1"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "5c9e4e49b238de7c0d4fc8c04da7be0c"
server_key_iv: "a14b4fc987f806cb6bed9dae13f7809a"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.327157 25878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "870a5d4205904f09a1da573b08858ca1"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "5c9e4e49b238de7c0d4fc8c04da7be0c"
server_key_iv: "a14b4fc987f806cb6bed9dae13f7809a"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.328389 25878 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:08.329125 25892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.329279 25878 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.329355 25878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "870a5d4205904f09a1da573b08858ca1"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "5c9e4e49b238de7c0d4fc8c04da7be0c"
server_key_iv: "a14b4fc987f806cb6bed9dae13f7809a"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.329406 25878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:08.332429 25878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:08.332623 25878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:08.332707 25878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:08.332903 25878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:08.333134 25878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:08.333163 25878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.333181 25878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:08.333194 25878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.338515 25878 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:46437
I20260924 21:17:08.338585 26005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:46437 every 8 connection(s)
I20260924 21:17:08.338871 25878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:17:08.340426 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 25878
I20260924 21:17:08.340533 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:08.340719 20943 external_mini_cluster.cc:1428] Setting key 76b464639812f4562765e2ea678d9426
I20260924 21:17:08.341733 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.339614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:48447 (local address 127.20.115.254:45701)
0924 21:17:08.339674 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.339676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.339888 (+ 212us) server_negotiation.cc:407] Connection header received
0924 21:17:08.340311 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.340312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.340318 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.340404 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.341564 (+ 1160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.341566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.341566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.341569 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.341583 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.341586 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.341629 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.341674 (+ 45us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.341685 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:17:08.341989 26008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.339595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45701 (local address 127.20.115.195:48447)
0924 21:17:08.339768 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.339805 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.340209 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.340448 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.340451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.340458 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:08.341443 (+ 985us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:08.341443 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:08.341470 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:08.341496 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:08.341629 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:08.341631 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:08.341633 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:17:08.341681 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 21:17:08.341724 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
I20260924 21:17:08.342372 26006 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45701
I20260924 21:17:08.342442 26006 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:08.342602 26006 heartbeater.cc:511] Master 127.20.115.254:45701 requested a full tablet report, sending...
I20260924 21:17:08.342941 25555 ts_manager.cc:194] Registered new tserver with Master: 870a5d4205904f09a1da573b08858ca1 (127.20.115.195:46437)
I20260924 21:17:08.352316 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:17:08.354665 25595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.353641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.20.115.254:45701)
0924 21:17:08.353689 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.353691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.353807 (+ 116us) server_negotiation.cc:407] Connection header received
0924 21:17:08.353876 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.353876 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.353880 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.353927 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.354506 (+ 579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.354507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.354507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.354509 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.354518 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.354527 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.354565 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.354611 (+ 46us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.354622 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 21:17:08.356057 25555 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:17:08.357628 25555 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:08.361485 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.359659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:43003 (local address 127.0.0.1:46090)
0924 21:17:08.359930 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.359940 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.359991 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.360273 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.360278 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.360285 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:08.360985 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:08.360985 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:08.361001 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:08.361029 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:08.361294 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:08.361300 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:08.361303 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:08.361388 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 21:17:08.361402 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260924 21:17:08.361541 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.359758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.20.115.194:43003)
0924 21:17:08.360134 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.360139 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.360148 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:08.360175 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.360177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.360186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.360259 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.361116 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.361123 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.361123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.361127 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.361157 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.361179 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.361265 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.361419 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.361438 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":76,"threads_started":1}
I20260924 21:17:08.361614 26014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.359659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:46437 (local address 127.0.0.1:52270)
0924 21:17:08.359836 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.359870 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.359918 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.360751 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.360752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.360754 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:08.361284 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:08.361284 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:08.361290 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:08.361300 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:08.361481 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:08.361481 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:08.361482 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:17:08.361532 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:17:08.361546 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":56,"threads_started":1}
I20260924 21:17:08.361649 26016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.359745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52270 (local address 127.20.115.195:46437)
0924 21:17:08.360611 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.360615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.360626 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:08.360649 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.360651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.360656 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.360749 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.361371 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.361378 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.361378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.361382 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.361412 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.361429 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.361483 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.361539 (+ 56us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.361554 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":835,"thread_start_us":124,"threads_started":1}
I20260924 21:17:08.361997 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.359893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:41947 (local address 127.0.0.1:60792)
0924 21:17:08.360030 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.360040 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.360073 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.360387 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.360388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.360391 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:08.361131 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:08.361131 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:08.361139 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:08.361151 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:08.361866 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:08.361866 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:08.361867 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:17:08.361930 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 21:17:08.361942 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260924 21:17:08.362052 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.360080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60792 (local address 127.20.115.193:41947)
0924 21:17:08.360245 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.360249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.360256 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:17:08.360276 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.360278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.360285 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.360383 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.361239 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.361247 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.361247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.361251 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.361285 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.361308 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.361401 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.361943 (+ 542us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.361958 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260924 21:17:08.362470 25940 tablet_service.cc:1467] Processing CreateTablet for tablet 14f9815d0637460189a9ade57346e315 (DEFAULT_TABLE table=test-table [id=6d9fedce626a424fb2ad1b4fcf3e5a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:08.362635 25809 tablet_service.cc:1467] Processing CreateTablet for tablet 14f9815d0637460189a9ade57346e315 (DEFAULT_TABLE table=test-table [id=6d9fedce626a424fb2ad1b4fcf3e5a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:08.362859 25940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14f9815d0637460189a9ade57346e315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:08.362998 25809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14f9815d0637460189a9ade57346e315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:08.362932 25678 tablet_service.cc:1467] Processing CreateTablet for tablet 14f9815d0637460189a9ade57346e315 (DEFAULT_TABLE table=test-table [id=6d9fedce626a424fb2ad1b4fcf3e5a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:08.363411 25678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14f9815d0637460189a9ade57346e315. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:08.365612 26020 tablet_bootstrap.cc:492] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: Bootstrap starting.
I20260924 21:17:08.366166 26020 tablet_bootstrap.cc:654] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:08.366483 26020 log.cc:824] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:08.366632 26021 tablet_bootstrap.cc:492] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: Bootstrap starting.
I20260924 21:17:08.366854 26022 tablet_bootstrap.cc:492] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437: Bootstrap starting.
I20260924 21:17:08.367211 26021 tablet_bootstrap.cc:654] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:08.367368 26020 tablet_bootstrap.cc:492] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: No bootstrap required, opened a new log
I20260924 21:17:08.367437 26020 ts_tablet_manager.cc:1397] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:08.367504 26022 tablet_bootstrap.cc:654] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:08.367517 26021 log.cc:824] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:08.367806 26022 log.cc:824] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:08.368221 26021 tablet_bootstrap.cc:492] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: No bootstrap required, opened a new log
I20260924 21:17:08.368290 26021 ts_tablet_manager.cc:1397] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:08.368584 26022 tablet_bootstrap.cc:492] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437: No bootstrap required, opened a new log
I20260924 21:17:08.368662 26022 ts_tablet_manager.cc:1397] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:08.370066 26020 raft_consensus.cc:348] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } }
I20260924 21:17:08.370251 26020 raft_consensus.cc:374] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:08.370290 26020 raft_consensus.cc:729] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9929f415835048e7a08c7d31d8d2826a, State: Initialized, Role: FOLLOWER
I20260924 21:17:08.370445 26020 consensus_queue.cc:260] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } }
I20260924 21:17:08.370689 26020 ts_tablet_manager.cc:1428] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260924 21:17:08.370800 25875 heartbeater.cc:503] Master 127.20.115.254:45701 was elected leader, sending a full tablet report...
I20260924 21:17:08.370850 26021 raft_consensus.cc:348] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } }
I20260924 21:17:08.370990 26021 raft_consensus.cc:374] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:08.371032 26021 raft_consensus.cc:729] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 870a5d4205904f09a1da573b08858ca1, State: Initialized, Role: FOLLOWER
I20260924 21:17:08.371054 26022 raft_consensus.cc:348] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } }
I20260924 21:17:08.371152 26021 consensus_queue.cc:260] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } }
I20260924 21:17:08.371204 26022 raft_consensus.cc:374] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:08.371240 26022 raft_consensus.cc:729] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0816fa32580d4283beb4f0501c8a5437, State: Initialized, Role: FOLLOWER
I20260924 21:17:08.371379 26021 ts_tablet_manager.cc:1428] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:17:08.371377 26022 consensus_queue.cc:260] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } }
I20260924 21:17:08.371518 26006 heartbeater.cc:503] Master 127.20.115.254:45701 was elected leader, sending a full tablet report...
I20260924 21:17:08.371600 26022 ts_tablet_manager.cc:1428] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 21:17:08.371695 25744 heartbeater.cc:503] Master 127.20.115.254:45701 was elected leader, sending a full tablet report...
I20260924 21:17:08.443735 26027 raft_consensus.cc:482] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:08.443854 26027 raft_consensus.cc:504] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } }
I20260924 21:17:08.444243 26027 leader_election.cc:288] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9929f415835048e7a08c7d31d8d2826a (127.20.115.194:43003), 0816fa32580d4283beb4f0501c8a5437 (127.20.115.193:41947)
I20260924 21:17:08.445600 26008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.444430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:41947 (local address 127.20.115.195:36931)
0924 21:17:08.444478 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.444490 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.444528 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.444731 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.444732 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.444735 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:08.445236 (+ 501us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:08.445236 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:08.445243 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:08.445275 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:08.445459 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:08.445460 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:08.445461 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:17:08.445514 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 21:17:08.445527 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260924 21:17:08.445654 26019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.444542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:36931 (local address 127.20.115.193:41947)
0924 21:17:08.444589 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.444592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.444601 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:08.444628 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.444628 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.444634 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.444697 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.445373 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.445375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.445375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.445379 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.445395 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.445399 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.445445 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.445537 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.445552 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:17:08.445796 26017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.444548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:38143 (local address 127.20.115.194:43003)
0924 21:17:08.444593 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.444595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.444813 (+ 218us) server_negotiation.cc:407] Connection header received
0924 21:17:08.444847 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.444848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.444854 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.444899 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.445526 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.445528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.445528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.445533 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.445548 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.445552 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.445608 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.445716 (+ 108us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.445731 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:17:08.446007 26029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.444435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:43003 (local address 127.20.115.195:38143)
0924 21:17:08.444668 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.444696 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.444750 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.444931 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.444932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.444935 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:08.445345 (+ 410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:08.445345 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:08.445350 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:08.445359 (+ 9us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:08.445616 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:08.445617 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:08.445617 (+ 0us) client_negotiation.cc:769] Sending connection context
0924 21:17:08.445917 (+ 300us) client_negotiation.cc:240] Negotiation successful
0924 21:17:08.445931 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":39,"thread_start_us":135,"threads_started":1}
I20260924 21:17:08.445990 25698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f9815d0637460189a9ade57346e315" candidate_uuid: "870a5d4205904f09a1da573b08858ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0816fa32580d4283beb4f0501c8a5437" is_pre_election: true
I20260924 21:17:08.446141 25698 raft_consensus.cc:2454] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 870a5d4205904f09a1da573b08858ca1 in term 0.
I20260924 21:17:08.446352 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f9815d0637460189a9ade57346e315" candidate_uuid: "870a5d4205904f09a1da573b08858ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9929f415835048e7a08c7d31d8d2826a" is_pre_election: true
I20260924 21:17:08.446383 25894 leader_election.cc:302] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0816fa32580d4283beb4f0501c8a5437, 870a5d4205904f09a1da573b08858ca1; no voters:
I20260924 21:17:08.446468 25829 raft_consensus.cc:2454] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 870a5d4205904f09a1da573b08858ca1 in term 0.
I20260924 21:17:08.446513 26027 raft_consensus.cc:2781] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:08.446573 26027 raft_consensus.cc:482] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:08.446604 26027 raft_consensus.cc:3037] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:08.447326 26027 raft_consensus.cc:504] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } }
I20260924 21:17:08.447467 26027 leader_election.cc:288] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [CANDIDATE]: Term 1 election: Requested vote from peers 9929f415835048e7a08c7d31d8d2826a (127.20.115.194:43003), 0816fa32580d4283beb4f0501c8a5437 (127.20.115.193:41947)
I20260924 21:17:08.447603 25698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f9815d0637460189a9ade57346e315" candidate_uuid: "870a5d4205904f09a1da573b08858ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0816fa32580d4283beb4f0501c8a5437"
I20260924 21:17:08.447641 25829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f9815d0637460189a9ade57346e315" candidate_uuid: "870a5d4205904f09a1da573b08858ca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9929f415835048e7a08c7d31d8d2826a"
I20260924 21:17:08.447687 25698 raft_consensus.cc:3037] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:08.447722 25829 raft_consensus.cc:3037] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:08.448414 25698 raft_consensus.cc:2454] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 870a5d4205904f09a1da573b08858ca1 in term 1.
I20260924 21:17:08.448439 25829 raft_consensus.cc:2454] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 870a5d4205904f09a1da573b08858ca1 in term 1.
I20260924 21:17:08.448604 25896 leader_election.cc:302] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 870a5d4205904f09a1da573b08858ca1, 9929f415835048e7a08c7d31d8d2826a; no voters:
I20260924 21:17:08.448717 26027 raft_consensus.cc:2781] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:08.448933 26027 raft_consensus.cc:686] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 1 LEADER]: Becoming Leader. State: Replica: 870a5d4205904f09a1da573b08858ca1, State: Running, Role: LEADER
I20260924 21:17:08.449030 26027 consensus_queue.cc:237] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } }
I20260924 21:17:08.449759 25556 catalog_manager.cc:5495] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 870a5d4205904f09a1da573b08858ca1 (127.20.115.195). New cstate: current_term: 1 leader_uuid: "870a5d4205904f09a1da573b08858ca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "870a5d4205904f09a1da573b08858ca1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46437 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 } health_report { overall_health: UNKNOWN } } }
I20260924 21:17:08.456429 26016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.455481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52274 (local address 127.20.115.195:46437)
0924 21:17:08.455530 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.455533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.455542 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:08.455635 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.455636 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.455653 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.455717 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.456220 (+ 503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.456221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.456221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.456226 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.456238 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.456242 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.456293 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.456343 (+ 50us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.456357 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:17:08.457970 25829 raft_consensus.cc:1261] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [term 1 FOLLOWER]: Refusing update from remote peer 870a5d4205904f09a1da573b08858ca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:08.458213 25698 raft_consensus.cc:1261] T 14f9815d0637460189a9ade57346e315 P 0816fa32580d4283beb4f0501c8a5437 [term 1 FOLLOWER]: Refusing update from remote peer 870a5d4205904f09a1da573b08858ca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:08.458356 26027 consensus_queue.cc:1035] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9929f415835048e7a08c7d31d8d2826a" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 43003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:08.458509 26030 consensus_queue.cc:1035] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0816fa32580d4283beb4f0501c8a5437" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:08.461858 26034 mvcc.cc:205] Tried to move back new op lower bound from 7333005838161760256 to 7333005838127874048. Current Snapshot: MvccSnapshot[applied={T|T < 7333005838161760256 or (T in {7333005838161760256})}]
I20260924 21:17:08.463280 25554 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:08.463353 25554 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:08.464039 25554 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af16c765a1d34a838207a0a9e03846b6: Sending DeleteTablet for 3 replicas of tablet 14f9815d0637460189a9ade57346e315
I20260924 21:17:08.464370 25940 tablet_service.cc:1514] Processing DeleteTablet for tablet 14f9815d0637460189a9ade57346e315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:08 UTC) from {username='slave'} at 127.0.0.1:52270
I20260924 21:17:08.464406 25809 tablet_service.cc:1514] Processing DeleteTablet for tablet 14f9815d0637460189a9ade57346e315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:08 UTC) from {username='slave'} at 127.0.0.1:46090
I20260924 21:17:08.464641 26043 tablet_replica.cc:330] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: stopping tablet replica
I20260924 21:17:08.464686 25678 tablet_service.cc:1514] Processing DeleteTablet for tablet 14f9815d0637460189a9ade57346e315 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:08 UTC) from {username='slave'} at 127.0.0.1:60792
I20260924 21:17:08.464745 26043 raft_consensus.cc:2229] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:08.464872 26043 raft_consensus.cc:2258] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:08.464923 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 25616
I20260924 21:17:08.466239 26042 tablet_replica.cc:330] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: stopping tablet replica
I20260924 21:17:08.466336 26042 raft_consensus.cc:2229] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:17:08.466450 26042 raft_consensus.cc:2258] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:08.467489 26043 ts_tablet_manager.cc:1905] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:08.469301 26043 ts_tablet_manager.cc:1918] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:08.469350 26043 log.cc:1196] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/14f9815d0637460189a9ade57346e315
I20260924 21:17:08.469609 26043 ts_tablet_manager.cc:1939] T 14f9815d0637460189a9ade57346e315 P 9929f415835048e7a08c7d31d8d2826a: Deleting consensus metadata
I20260924 21:17:08.469739 26042 ts_tablet_manager.cc:1905] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:08.470045 25543 catalog_manager.cc:4841] TS 9929f415835048e7a08c7d31d8d2826a (127.20.115.194:43003): tablet 14f9815d0637460189a9ade57346e315 (table test-table [id=6d9fedce626a424fb2ad1b4fcf3e5a5b]) successfully deleted
I20260924 21:17:08.471261 26042 ts_tablet_manager.cc:1918] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:08.471349 26042 log.cc:1196] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/14f9815d0637460189a9ade57346e315
I20260924 21:17:08.471601 26042 ts_tablet_manager.cc:1939] T 14f9815d0637460189a9ade57346e315 P 870a5d4205904f09a1da573b08858ca1: Deleting consensus metadata
I20260924 21:17:08.471963 25541 catalog_manager.cc:4841] TS 870a5d4205904f09a1da573b08858ca1 (127.20.115.195:46437): tablet 14f9815d0637460189a9ade57346e315 (table test-table [id=6d9fedce626a424fb2ad1b4fcf3e5a5b]) successfully deleted
W20260924 21:17:08.473798 25541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:41947 (error 108)
I20260924 21:17:08.474036 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 25747
I20260924 21:17:08.474705 26018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.474177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:41947 (local address 127.0.0.1:60808)
0924 21:17:08.474623 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.474666 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:41947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425}
W20260924 21:17:08.474831 25541 catalog_manager.cc:4584] TS 0816fa32580d4283beb4f0501c8a5437 (127.20.115.193:41947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14f9815d0637460189a9ade57346e315: Network error: Client connection negotiation failed: client connection to 127.20.115.193:41947: connect: Connection refused (error 111)
I20260924 21:17:08.479490 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 25878
I20260924 21:17:08.484084 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 25525
2026-09-24T21:17:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (637 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260924 21:17:08.503603 20943 mini_oidc.cc:152] Starting JWKS server
I20260924 21:17:08.504582 20943 webserver.cc:460] Webserver started at https://127.0.0.1:38619/ using document root <none> and password file <none>
I20260924 21:17:08.504706 20943 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 21:17:08.504940 20943 webserver.cc:460] Webserver started at http://127.0.0.1:42671/ using document root <none> and password file <none>
2026-09-24T21:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:08Z Disabled control of system clock
I20260924 21:17:08.512365 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33601
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:37213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:33601
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38619/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/testchainca.cert with env {}
W20260924 21:17:08.590528 26051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:08.590713 26051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:08.590734 26051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:08.591877 26051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:08.591915 26051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:08.591928 26051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:08.591940 26051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:08.591992 26051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 21:17:08.592008 26051 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 21:17:08.592031 26051 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38619/jwks.json
I20260924 21:17:08.593537 26051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:33601
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38619/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:08.593765 26051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:08.593995 26051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:08.598793 26059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:08.598757 26057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:08.598800 26051 server_base.cc:1034] running on GCE node
W20260924 21:17:08.598763 26056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:08.599282 26051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:08.599555 26051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:08.600695 26051 hybrid_clock.cc:648] HybridClock initialized: now 1790284628600677 us; error 40 us; skew 500 ppm
I20260924 21:17:08.601871 26051 webserver.cc:460] Webserver started at http://127.20.115.254:46585/ using document root <none> and password file <none>
I20260924 21:17:08.602108 26051 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:08.602154 26051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:08.602303 26051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:08.603284 26051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "fc6be091b7df48c28c84fc0be5726954"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "eeb9b36844e1366cbb8612a877d00dac"
server_key_iv: "3de628402c300ed2ed7ea60efdeb9097"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.603641 26051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "fc6be091b7df48c28c84fc0be5726954"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "eeb9b36844e1366cbb8612a877d00dac"
server_key_iv: "3de628402c300ed2ed7ea60efdeb9097"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.604939 26051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:08.605667 26065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.605851 26051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:08.605917 26051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "fc6be091b7df48c28c84fc0be5726954"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "eeb9b36844e1366cbb8612a877d00dac"
server_key_iv: "3de628402c300ed2ed7ea60efdeb9097"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.605991 26051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:08.618480 26051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:08.618721 26051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:08.618814 26051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:08.622781 26051 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:33601
I20260924 21:17:08.622853 26117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:33601 every 8 connection(s)
I20260924 21:17:08.623144 26051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:08.624027 26118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:08.626273 26118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954: Bootstrap starting.
I20260924 21:17:08.626510 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 26051
I20260924 21:17:08.626595 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:08.626776 20943 external_mini_cluster.cc:1428] Setting key c49399426ecb1c4691ac38825dfa2786
I20260924 21:17:08.626986 26118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:08.627326 26118 log.cc:824] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:08.628077 26118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954: No bootstrap required, opened a new log
I20260924 21:17:08.630365 26118 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6be091b7df48c28c84fc0be5726954" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33601 } }
I20260924 21:17:08.630497 26118 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:08.630520 26118 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc6be091b7df48c28c84fc0be5726954, State: Initialized, Role: FOLLOWER
I20260924 21:17:08.630647 26118 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6be091b7df48c28c84fc0be5726954" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33601 } }
I20260924 21:17:08.630719 26118 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:08.630784 26118 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:08.630836 26118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:08.631321 26118 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6be091b7df48c28c84fc0be5726954" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33601 } }
I20260924 21:17:08.631443 26118 leader_election.cc:302] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc6be091b7df48c28c84fc0be5726954; no voters:
I20260924 21:17:08.631592 26118 leader_election.cc:288] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:08.631851 26123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:08.631862 26118 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:08.632015 26123 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [term 1 LEADER]: Becoming Leader. State: Replica: fc6be091b7df48c28c84fc0be5726954, State: Running, Role: LEADER
I20260924 21:17:08.632091 26123 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6be091b7df48c28c84fc0be5726954" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33601 } }
I20260924 21:17:08.632440 26125 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc6be091b7df48c28c84fc0be5726954. Latest consensus state: current_term: 1 leader_uuid: "fc6be091b7df48c28c84fc0be5726954" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6be091b7df48c28c84fc0be5726954" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33601 } } }
I20260924 21:17:08.632517 26125 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:08.632669 26124 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc6be091b7df48c28c84fc0be5726954" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc6be091b7df48c28c84fc0be5726954" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33601 } } }
I20260924 21:17:08.632715 26124 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:08.632751 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.627264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38876 (local address 127.20.115.254:33601)
0924 21:17:08.627474 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.627485 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.627521 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:17:08.628064 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.628073 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.628145 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.628299 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.628475 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.629835 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.630689 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.631175 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.631305 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:08.631970 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.631973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.631974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.632159 (+ 185us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.632197 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.632213 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.632266 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.632461 (+ 195us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.632537 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":80,"threads_started":1}
I20260924 21:17:08.632930 26132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:08.633360 26132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:08.633792 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:17:08.634773 26132 catalog_manager.cc:1347] Generated new cluster ID: 822a23bdbb0743febf9499d5d6245ecb
I20260924 21:17:08.634835 26132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:08.649062 26132 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:08.649633 26132 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:08.652522 26132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc6be091b7df48c28c84fc0be5726954: Generated new TSK 0
I20260924 21:17:08.652673 26132 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 21:17:08.673580 26148 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260924 21:17:08.673671 26148 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 21:17:08.673880 26121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.658845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38884 (local address 127.20.115.254:33601)
0924 21:17:08.658886 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.658888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.658986 (+ 98us) server_negotiation.cc:407] Connection header received
0924 21:17:08.659058 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.659059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.659069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.659147 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0924 21:17:08.659249 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.660009 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.660716 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.661054 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.661122 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:08.661296 (+ 174us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 21:17:08.673688 (+ 12392us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 21:17:08.673814 (+ 126us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38884: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260924 21:17:08.673959 26121 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38884: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260924 21:17:08.674234 26147 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:08.658776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33601 (local address 127.0.0.1:38884)
0924 21:17:08.658920 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.658929 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.658980 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.659147 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.659148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.659149 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0924 21:17:08.659194 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:08.659195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.660086 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.660086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:08.660642 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:08.660643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.661133 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.661134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:08.661178 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:08.661194 (+ 16us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 21:17:08.673833 (+ 12639us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0924 21:17:08.673914 (+ 81us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.115.254:33601: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.115.254:33601: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38619/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260924 21:17:08.674762 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 26051
2026-09-24T21:17:08Z chronyd exiting
I20260924 21:17:08.712484 20943 test_util.cc:175] -----------------------------------------------
I20260924 21:17:08.712567 20943 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790284606872664-20943-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (209 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260924 21:17:08.713400 20943 mini_oidc.cc:152] Starting JWKS server
I20260924 21:17:08.714076 20943 webserver.cc:460] Webserver started at https://127.0.0.1:39843/ using document root <none> and password file <none>
I20260924 21:17:08.714115 20943 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 21:17:08.714298 20943 webserver.cc:460] Webserver started at http://127.0.0.1:43043/ using document root <none> and password file <none>
2026-09-24T21:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:08Z Disabled control of system clock
I20260924 21:17:08.721617 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:42675
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:42675
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39843/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/testchainca.cert with env {}
W20260924 21:17:08.799667 26154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:08.799901 26154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:08.799937 26154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:08.801120 26154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:08.801191 26154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:08.801213 26154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:08.801242 26154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:08.801275 26154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 21:17:08.801296 26154 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 21:17:08.801316 26154 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39843/jwks.json
I20260924 21:17:08.802767 26154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:42675
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:42675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39843/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:08.803035 26154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:08.803246 26154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:08.807861 26162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:08.807871 26160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:08.807870 26159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:08.808097 26154 server_base.cc:1034] running on GCE node
I20260924 21:17:08.808331 26154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:08.808620 26154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:08.809770 26154 hybrid_clock.cc:648] HybridClock initialized: now 1790284628809753 us; error 29 us; skew 500 ppm
I20260924 21:17:08.810882 26154 webserver.cc:460] Webserver started at http://127.20.115.254:36057/ using document root <none> and password file <none>
I20260924 21:17:08.811112 26154 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:08.811180 26154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:08.811308 26154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:08.812312 26154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "547271987ffc407b8b9e3fae3902c16a"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "ff9a590768b579fa3617a1265782492c"
server_key_iv: "97a0ebce96078c9335252f4aa43e4314"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.812671 26154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "547271987ffc407b8b9e3fae3902c16a"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "ff9a590768b579fa3617a1265782492c"
server_key_iv: "97a0ebce96078c9335252f4aa43e4314"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.813977 26154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:08.814659 26168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:08.814826 26154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:08.814932 26154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "547271987ffc407b8b9e3fae3902c16a"
format_stamp: "Formatted at 2026-09-24 21:17:08 on dist-test-slave-cxpd"
server_key: "ff9a590768b579fa3617a1265782492c"
server_key_iv: "97a0ebce96078c9335252f4aa43e4314"
server_key_version: "encryptionkey@0"
I20260924 21:17:08.814996 26154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:08.831499 26154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:08.831773 26154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:08.831920 26154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:08.835736 26154 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:42675
I20260924 21:17:08.835765 26220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:42675 every 8 connection(s)
I20260924 21:17:08.836079 26154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:08.836689 26221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:08.839272 26221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a: Bootstrap starting.
I20260924 21:17:08.839771 26221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:08.839998 26221 log.cc:824] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:08.840583 26221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a: No bootstrap required, opened a new log
I20260924 21:17:08.842125 26221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547271987ffc407b8b9e3fae3902c16a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42675 } }
I20260924 21:17:08.842254 26221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:08.842275 26221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547271987ffc407b8b9e3fae3902c16a, State: Initialized, Role: FOLLOWER
I20260924 21:17:08.842410 26221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547271987ffc407b8b9e3fae3902c16a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42675 } }
I20260924 21:17:08.842494 26221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:08.842517 26221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:08.842545 26221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:08.843006 26221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547271987ffc407b8b9e3fae3902c16a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42675 } }
I20260924 21:17:08.843104 26221 leader_election.cc:302] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 547271987ffc407b8b9e3fae3902c16a; no voters:
I20260924 21:17:08.843215 26221 leader_election.cc:288] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:08.843284 26224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:08.843446 26224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [term 1 LEADER]: Becoming Leader. State: Replica: 547271987ffc407b8b9e3fae3902c16a, State: Running, Role: LEADER
I20260924 21:17:08.843489 26221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:08.843535 26224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547271987ffc407b8b9e3fae3902c16a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42675 } }
I20260924 21:17:08.843892 26225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "547271987ffc407b8b9e3fae3902c16a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547271987ffc407b8b9e3fae3902c16a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42675 } } }
I20260924 21:17:08.843979 26225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:08.844079 26226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 547271987ffc407b8b9e3fae3902c16a. Latest consensus state: current_term: 1 leader_uuid: "547271987ffc407b8b9e3fae3902c16a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547271987ffc407b8b9e3fae3902c16a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42675 } } }
I20260924 21:17:08.844147 26226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:08.844269 26233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:08.844954 26233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:08.845582 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 26154
I20260924 21:17:08.845687 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:08.845857 20943 external_mini_cluster.cc:1428] Setting key d5b0732d429f53d01c3d8b0c7da86306
I20260924 21:17:08.847152 26233 catalog_manager.cc:1347] Generated new cluster ID: 9c5c4447e6cd4c1987f78486da1da868
I20260924 21:17:08.847206 26233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:08.851320 26233 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:08.851882 26233 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:08.851907 26243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.846322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35470 (local address 127.20.115.254:42675)
0924 21:17:08.846509 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.846517 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.846558 (+ 41us) server_negotiation.cc:407] Connection header received
0924 21:17:08.846640 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.846646 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.846692 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.846796 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:08.847055 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.848406 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.849559 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.850143 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.850253 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:08.851053 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:08.851057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:08.851057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:08.851266 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:08.851319 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:08.851334 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:08.851398 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:08.851505 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 21:17:08.851583 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260924 21:17:08.852865 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:17:08.857609 26233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 547271987ffc407b8b9e3fae3902c16a: Generated new TSK 0
I20260924 21:17:08.857776 26233 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260924 21:17:08.879567 26251 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260924 21:17:08.879654 26251 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260924 21:17:08.879947 26243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:08.863236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.20.115.254:42675)
0924 21:17:08.863275 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:08.863277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:08.863344 (+ 67us) server_negotiation.cc:407] Connection header received
0924 21:17:08.863407 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:08.863408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:08.863416 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:08.863466 (+ 50us) server_negotiation.cc:226] Negotiated authn=JWT
0924 21:17:08.863582 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.864035 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.865186 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.865372 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.865422 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:08.865600 (+ 178us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 21:17:08.879774 (+ 14174us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 21:17:08.879898 (+ 124us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35472: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":22}
W20260924 21:17:08.880014 26243 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35472: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260924 21:17:08.880271 26250 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:08.863138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:42675 (local address 127.0.0.1:35472)
0924 21:17:08.863271 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:08.863282 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:08.863345 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:08.863475 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:08.863476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:08.863478 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 21:17:08.863527 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:08.863528 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.864126 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.864127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:08.865112 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:08.865114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:08.865451 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:08.865452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:08.865511 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:08.865515 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 21:17:08.879896 (+ 14381us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0924 21:17:08.880031 (+ 135us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.115.254:42675: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39843/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":90,"threads_started":1}
I20260924 21:17:08.880594 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 26154
2026-09-24T21:17:08Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (204 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260924 21:17:08.917433 20943 mini_oidc.cc:152] Starting JWKS server
I20260924 21:17:08.918118 20943 webserver.cc:460] Webserver started at https://127.0.0.1:40441/ using document root <none> and password file <none>
I20260924 21:17:08.918187 20943 mini_oidc.cc:189] Starting OIDC Discovery server
I20260924 21:17:08.918360 20943 webserver.cc:460] Webserver started at http://127.0.0.1:35913/ using document root <none> and password file <none>
2026-09-24T21:17:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:08Z Disabled control of system clock
I20260924 21:17:08.926062 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:39443
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:38975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:39443
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40441/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 21:17:09.004555 26257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:09.004745 26257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:09.004765 26257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:09.005936 26257 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:09.005977 26257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:09.005990 26257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:09.006002 26257 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:09.006014 26257 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 21:17:09.006026 26257 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 21:17:09.006085 26257 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40441/jwks.json
I20260924 21:17:09.007400 26257 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:38975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:39443
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:39443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40441/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:09.007648 26257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:09.007930 26257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:09.012580 26265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:09.012705 26257 server_base.cc:1034] running on GCE node
W20260924 21:17:09.012615 26263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:09.012594 26262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:09.013046 26257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:09.013326 26257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:09.014493 26257 hybrid_clock.cc:648] HybridClock initialized: now 1790284629014482 us; error 30 us; skew 500 ppm
I20260924 21:17:09.015702 26257 webserver.cc:460] Webserver started at http://127.20.115.254:36425/ using document root <none> and password file <none>
I20260924 21:17:09.015929 26257 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:09.016004 26257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:09.016157 26257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:09.017237 26257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "09dfb034b54f4c799a022fb1457b8b88"
format_stamp: "Formatted at 2026-09-24 21:17:09 on dist-test-slave-cxpd"
server_key: "8ff47d49b9ac2c59e2ce427080e17a73"
server_key_iv: "92f2d087e6b2eff0f37c4f994bbf271a"
server_key_version: "encryptionkey@0"
I20260924 21:17:09.017577 26257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "09dfb034b54f4c799a022fb1457b8b88"
format_stamp: "Formatted at 2026-09-24 21:17:09 on dist-test-slave-cxpd"
server_key: "8ff47d49b9ac2c59e2ce427080e17a73"
server_key_iv: "92f2d087e6b2eff0f37c4f994bbf271a"
server_key_version: "encryptionkey@0"
I20260924 21:17:09.018915 26257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:09.019622 26271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:09.019799 26257 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:09.019896 26257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "09dfb034b54f4c799a022fb1457b8b88"
format_stamp: "Formatted at 2026-09-24 21:17:09 on dist-test-slave-cxpd"
server_key: "8ff47d49b9ac2c59e2ce427080e17a73"
server_key_iv: "92f2d087e6b2eff0f37c4f994bbf271a"
server_key_version: "encryptionkey@0"
I20260924 21:17:09.020001 26257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:09.037660 26257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:09.037943 26257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:09.038100 26257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:09.041813 26257 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:39443
I20260924 21:17:09.041847 26323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:39443 every 8 connection(s)
I20260924 21:17:09.042193 26257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:09.042857 26324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:09.045524 26324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88: Bootstrap starting.
I20260924 21:17:09.046095 26324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:09.046373 26324 log.cc:824] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:09.047011 26324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88: No bootstrap required, opened a new log
I20260924 21:17:09.048534 26324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dfb034b54f4c799a022fb1457b8b88" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39443 } }
I20260924 21:17:09.048664 26324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:09.048686 26324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09dfb034b54f4c799a022fb1457b8b88, State: Initialized, Role: FOLLOWER
I20260924 21:17:09.048892 26324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dfb034b54f4c799a022fb1457b8b88" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39443 } }
I20260924 21:17:09.048974 26324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:09.048995 26324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:09.049068 26324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:09.049571 26324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dfb034b54f4c799a022fb1457b8b88" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39443 } }
I20260924 21:17:09.049675 26324 leader_election.cc:302] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09dfb034b54f4c799a022fb1457b8b88; no voters:
I20260924 21:17:09.049841 26324 leader_election.cc:288] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:09.049907 26327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:09.050048 26327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [term 1 LEADER]: Becoming Leader. State: Replica: 09dfb034b54f4c799a022fb1457b8b88, State: Running, Role: LEADER
I20260924 21:17:09.050086 26324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:09.050141 26327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dfb034b54f4c799a022fb1457b8b88" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39443 } }
I20260924 21:17:09.050500 26328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09dfb034b54f4c799a022fb1457b8b88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dfb034b54f4c799a022fb1457b8b88" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39443 } } }
I20260924 21:17:09.050560 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 26257
I20260924 21:17:09.050586 26328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:09.050664 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:09.050690 26329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09dfb034b54f4c799a022fb1457b8b88. Latest consensus state: current_term: 1 leader_uuid: "09dfb034b54f4c799a022fb1457b8b88" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09dfb034b54f4c799a022fb1457b8b88" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39443 } } }
I20260924 21:17:09.050747 26329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:09.050875 20943 external_mini_cluster.cc:1428] Setting key a5de576393860673c8e4685aaacb5059
I20260924 21:17:09.051186 26335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:09.051657 26335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:09.053474 26335 catalog_manager.cc:1347] Generated new cluster ID: 61730fd3fbe243f992bfd3ad7f1ad869
I20260924 21:17:09.053526 26335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:09.056389 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:09.051505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55676 (local address 127.20.115.254:39443)
0924 21:17:09.051922 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:09.051931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:09.051967 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:17:09.052089 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:09.052096 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:09.052169 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:09.052310 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:09.052485 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.053762 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.054548 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.054965 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.055100 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:09.055745 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:09.055748 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:09.055749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:09.055920 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:09.055950 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:09.055961 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:09.056036 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:09.056114 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 21:17:09.056188 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":65,"threads_started":1}
I20260924 21:17:09.057206 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:17:09.076162 26335 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:09.076634 26335 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:09.083247 26335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09dfb034b54f4c799a022fb1457b8b88: Generated new TSK 0
I20260924 21:17:09.083433 26335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:17:09.099221 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:09.095154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.20.115.254:39443)
0924 21:17:09.095216 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:09.095219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:09.095314 (+ 95us) server_negotiation.cc:407] Connection header received
0924 21:17:09.095388 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:09.095389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:09.095398 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:09.095456 (+ 58us) server_negotiation.cc:226] Negotiated authn=JWT
0924 21:17:09.095597 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.096096 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.097276 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.097479 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.097535 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:09.097742 (+ 207us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 21:17:09.099005 (+ 1263us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 21:17:09.099134 (+ 129us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55686: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260924 21:17:09.099280 26344 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55686: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260924 21:17:09.099471 26353 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:09.095088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39443 (local address 127.0.0.1:55686)
0924 21:17:09.095243 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:09.095254 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:09.095322 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:09.095473 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:09.095474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:09.095476 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 21:17:09.095536 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:09.095537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.096176 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.096177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:09.097173 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:09.097176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.097559 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.097560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:09.097618 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:09.097624 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 21:17:09.099108 (+ 1484us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0924 21:17:09.099209 (+ 101us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.115.254:39443: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40441/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":106,"threads_started":1}
I20260924 21:17:09.099685 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 26257
E20260924 21:17:09.110641 26354 webserver.cc:533] Webserver: sslize failed:
E20260924 21:17:09.110749 26354 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-24T21:17:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (220 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-24T21:17:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:09Z Disabled control of system clock
I20260924 21:17:09.145306 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43565
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:44649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:43565
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/testchainca.cert with env {}
W20260924 21:17:09.224519 26358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:09.224772 26358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:09.224824 26358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:09.226027 26358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:09.226104 26358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:09.226125 26358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:09.226145 26358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:09.226176 26358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260924 21:17:09.226198 26358 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260924 21:17:09.226219 26358 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260924 21:17:09.227717 26358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:44649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:43565
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:09.227998 26358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:09.228222 26358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:09.233112 26364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:09.233117 26363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:09.233198 26366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:09.233260 26358 server_base.cc:1034] running on GCE node
I20260924 21:17:09.233598 26358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:09.233891 26358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:09.235069 26358 hybrid_clock.cc:648] HybridClock initialized: now 1790284629235032 us; error 51 us; skew 500 ppm
I20260924 21:17:09.236375 26358 webserver.cc:460] Webserver started at http://127.20.115.254:35895/ using document root <none> and password file <none>
I20260924 21:17:09.236615 26358 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:09.236661 26358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:09.236824 26358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:09.237898 26358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "b80c675eb66a43ce8004518c637e39ee"
format_stamp: "Formatted at 2026-09-24 21:17:09 on dist-test-slave-cxpd"
server_key: "e7d34ca59346d48b2ae2eb28b2bb74d7"
server_key_iv: "e8446394fdcf446333cb549d8a8c073c"
server_key_version: "encryptionkey@0"
I20260924 21:17:09.238270 26358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "b80c675eb66a43ce8004518c637e39ee"
format_stamp: "Formatted at 2026-09-24 21:17:09 on dist-test-slave-cxpd"
server_key: "e7d34ca59346d48b2ae2eb28b2bb74d7"
server_key_iv: "e8446394fdcf446333cb549d8a8c073c"
server_key_version: "encryptionkey@0"
I20260924 21:17:09.239511 26358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:17:09.240208 26372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:09.240412 26358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:09.240505 26358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "b80c675eb66a43ce8004518c637e39ee"
format_stamp: "Formatted at 2026-09-24 21:17:09 on dist-test-slave-cxpd"
server_key: "e7d34ca59346d48b2ae2eb28b2bb74d7"
server_key_iv: "e8446394fdcf446333cb549d8a8c073c"
server_key_version: "encryptionkey@0"
I20260924 21:17:09.240576 26358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:09.250967 26358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:09.251250 26358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:09.251394 26358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:09.255271 26358 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:43565
I20260924 21:17:09.255300 26424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:43565 every 8 connection(s)
I20260924 21:17:09.255643 26358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:09.256569 26425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:09.258888 26425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee: Bootstrap starting.
I20260924 21:17:09.259487 26425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:09.259609 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 26358
I20260924 21:17:09.259716 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:09.259833 26425 log.cc:824] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:09.259931 20943 external_mini_cluster.cc:1428] Setting key cdf9668fb96cfea100c8c10298915efd
I20260924 21:17:09.260568 26425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee: No bootstrap required, opened a new log
I20260924 21:17:09.262466 26425 raft_consensus.cc:348] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80c675eb66a43ce8004518c637e39ee" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43565 } }
I20260924 21:17:09.262642 26425 raft_consensus.cc:374] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:09.262689 26425 raft_consensus.cc:729] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80c675eb66a43ce8004518c637e39ee, State: Initialized, Role: FOLLOWER
I20260924 21:17:09.262825 26425 consensus_queue.cc:260] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80c675eb66a43ce8004518c637e39ee" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43565 } }
I20260924 21:17:09.262926 26425 raft_consensus.cc:388] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:09.262971 26425 raft_consensus.cc:482] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:09.263019 26425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:09.263568 26425 raft_consensus.cc:504] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80c675eb66a43ce8004518c637e39ee" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43565 } }
I20260924 21:17:09.263682 26425 leader_election.cc:302] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b80c675eb66a43ce8004518c637e39ee; no voters:
I20260924 21:17:09.263834 26425 leader_election.cc:288] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:09.263887 26430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:09.264003 26430 raft_consensus.cc:686] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [term 1 LEADER]: Becoming Leader. State: Replica: b80c675eb66a43ce8004518c637e39ee, State: Running, Role: LEADER
I20260924 21:17:09.264092 26425 sys_catalog.cc:564] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:09.264096 26430 consensus_queue.cc:237] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80c675eb66a43ce8004518c637e39ee" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43565 } }
I20260924 21:17:09.264452 26430 sys_catalog.cc:455] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader b80c675eb66a43ce8004518c637e39ee. Latest consensus state: current_term: 1 leader_uuid: "b80c675eb66a43ce8004518c637e39ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80c675eb66a43ce8004518c637e39ee" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43565 } } }
I20260924 21:17:09.264539 26430 sys_catalog.cc:458] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:09.264849 26432 sys_catalog.cc:455] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b80c675eb66a43ce8004518c637e39ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b80c675eb66a43ce8004518c637e39ee" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43565 } } }
I20260924 21:17:09.264925 26432 sys_catalog.cc:458] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee [sys.catalog]: This master's current role is: LEADER
W20260924 21:17:09.266088 26446 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:17:09.266146 26446 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:17:09.266245 26439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:09.266428 26439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:09.266989 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:09.260432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49506 (local address 127.20.115.254:43565)
0924 21:17:09.260650 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:09.260659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:09.260690 (+ 31us) server_negotiation.cc:407] Connection header received
0924 21:17:09.261729 (+ 1039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:09.261737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:09.261799 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:09.261951 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:09.262191 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.263542 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.264632 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.265294 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.265390 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:09.266381 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:09.266385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:09.266385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:09.266562 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:09.266593 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:09.266603 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:09.266671 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:09.266747 (+ 76us) server_negotiation.cc:299] Negotiation successful
0924 21:17:09.266827 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":77,"threads_started":1}
I20260924 21:17:09.267937 26439 catalog_manager.cc:1347] Generated new cluster ID: e9ca75996c8948b3a96b2e53092e52d1
I20260924 21:17:09.267983 26439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:09.267944 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:17:09.281519 26439 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:09.282186 26439 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:09.284767 26439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b80c675eb66a43ce8004518c637e39ee: Generated new TSK 0
I20260924 21:17:09.284947 26439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:17:09.412729 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:09.292469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49518 (local address 127.20.115.254:43565)
0924 21:17:09.292512 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:09.292514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:09.292625 (+ 111us) server_negotiation.cc:407] Connection header received
0924 21:17:09.292715 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:09.292716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:09.292728 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:09.292883 (+ 155us) server_negotiation.cc:226] Negotiated authn=JWT
0924 21:17:09.293033 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.293521 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.294323 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.294540 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.294615 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:09.294806 (+ 191us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0924 21:17:09.412405 (+117599us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 21:17:09.412628 (+ 223us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49518: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 21:17:09.412842 26429 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49518: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260924 21:17:09.412936 26454 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:09.292386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43565 (local address 127.0.0.1:49518)
0924 21:17:09.292556 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:09.292565 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:09.292643 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:09.292885 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:09.292886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:09.292888 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0924 21:17:09.292950 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:09.292952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.293622 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.293623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:09.294226 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:09.294228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:09.294619 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:09.294620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:09.294678 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:09.294683 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0924 21:17:09.412577 (+117894us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0924 21:17:09.412715 (+ 138us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.115.254:43565: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":78,"threads_started":1}
I20260924 21:17:09.413326 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 26358
2026-09-24T21:17:09Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (293 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (87 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (86 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:09 dist-test-slave-cxpd krb5kdc[26466](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:09 dist-test-slave-cxpd krb5kdc[26466](info): set up 2 sockets
Sep 24 21:17:09 dist-test-slave-cxpd krb5kdc[26466](info): commencing operation
krb5kdc: starting...
W20260924 21:17:10.609084 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.004s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:10 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:17:10.724347 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:34383
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:34383
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:10.801888 26479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:10.802117 26479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:10.802177 26479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:10.803320 26479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:10.803390 26479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:10.803416 26479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:10.803447 26479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:10.803470 26479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:10.805085 26479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:34383
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:34383
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:10.805346 26479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:10.805575 26479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:10.810190 26487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:10.810232 26484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:10.810233 26485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:10.810411 26479 server_base.cc:1034] running on GCE node
I20260924 21:17:10.810644 26479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:10.810940 26479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:10.812073 26479 hybrid_clock.cc:648] HybridClock initialized: now 1790284630812062 us; error 31 us; skew 500 ppm
Sep 24 21:17:10 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284630, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:10.814447 26479 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:17:10.814901 26479 webserver.cc:460] Webserver started at http://127.20.115.254:35171/ using document root <none> and password file <none>
I20260924 21:17:10.815126 26479 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:10.815194 26479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:10.815346 26479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:10.816360 26479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "ea8958e1c5254ebb935bd1e1655d69ef"
format_stamp: "Formatted at 2026-09-24 21:17:10 on dist-test-slave-cxpd"
server_key: "e6ecfa8f9292e77db222012e427104de"
server_key_iv: "85aadb960cadfa0957044fcdc4e27327"
server_key_version: "encryptionkey@0"
I20260924 21:17:10.816713 26479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "ea8958e1c5254ebb935bd1e1655d69ef"
format_stamp: "Formatted at 2026-09-24 21:17:10 on dist-test-slave-cxpd"
server_key: "e6ecfa8f9292e77db222012e427104de"
server_key_iv: "85aadb960cadfa0957044fcdc4e27327"
server_key_version: "encryptionkey@0"
I20260924 21:17:10.818018 26479 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:10.818674 26494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:10.818830 26479 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:10.818924 26479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "ea8958e1c5254ebb935bd1e1655d69ef"
format_stamp: "Formatted at 2026-09-24 21:17:10 on dist-test-slave-cxpd"
server_key: "e6ecfa8f9292e77db222012e427104de"
server_key_iv: "85aadb960cadfa0957044fcdc4e27327"
server_key_version: "encryptionkey@0"
I20260924 21:17:10.819005 26479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:10.830752 26479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:10.832440 26479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:10.832599 26479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:10.836421 26479 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:34383
I20260924 21:17:10.836457 26546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:34383 every 8 connection(s)
I20260924 21:17:10.836853 26479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:10.837651 26547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:10.838436 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 26479
I20260924 21:17:10.838557 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:10.838744 20943 external_mini_cluster.cc:1428] Setting key ccc6d0a5b8b8cd5798082b04685b2ef4
I20260924 21:17:10.840719 26547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef: Bootstrap starting.
I20260924 21:17:10.841425 26547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:10.841780 26547 log.cc:824] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:10.842667 26547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef: No bootstrap required, opened a new log
I20260924 21:17:10.845446 26547 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8958e1c5254ebb935bd1e1655d69ef" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34383 } }
I20260924 21:17:10.845645 26547 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:10.845688 26547 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea8958e1c5254ebb935bd1e1655d69ef, State: Initialized, Role: FOLLOWER
I20260924 21:17:10.845839 26547 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8958e1c5254ebb935bd1e1655d69ef" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34383 } }
I20260924 21:17:10.845916 26547 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:10.845952 26547 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:10.846004 26547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [term 0 FOLLOWER]: Advancing to term 1
Sep 24 21:17:10 dist-test-slave-cxpd krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:10.846760 26547 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8958e1c5254ebb935bd1e1655d69ef" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34383 } }
I20260924 21:17:10.846876 26547 leader_election.cc:302] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea8958e1c5254ebb935bd1e1655d69ef; no voters:
I20260924 21:17:10.847043 26547 leader_election.cc:288] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:10.847141 26552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:10.847311 26547 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:10.847332 26552 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [term 1 LEADER]: Becoming Leader. State: Replica: ea8958e1c5254ebb935bd1e1655d69ef, State: Running, Role: LEADER
I20260924 21:17:10.847501 26552 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8958e1c5254ebb935bd1e1655d69ef" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34383 } }
I20260924 21:17:10.847945 26553 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea8958e1c5254ebb935bd1e1655d69ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8958e1c5254ebb935bd1e1655d69ef" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34383 } } }
I20260924 21:17:10.848044 26553 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:10.847963 26554 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea8958e1c5254ebb935bd1e1655d69ef. Latest consensus state: current_term: 1 leader_uuid: "ea8958e1c5254ebb935bd1e1655d69ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea8958e1c5254ebb935bd1e1655d69ef" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34383 } } }
I20260924 21:17:10.848133 26554 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:10.848814 26561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:10.850073 26561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:10.851670 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:10.839233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53128 (local address 127.20.115.254:34383)
0924 21:17:10.839395 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:10.839404 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:10.839435 (+ 31us) server_negotiation.cc:407] Connection header received
0924 21:17:10.839852 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:10.839860 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:10.839919 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:10.840059 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:10.840962 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:10.842451 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:10.843619 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:10.844313 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:10.844424 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:10.847311 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:10.847316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:10.847316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:10.847327 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:10.849872 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:10.850282 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:10.850285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:10.850286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:10.850378 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:10.850576 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:10.850577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:10.850577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:10.850850 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:10.851013 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:10.851325 (+ 312us) server_negotiation.cc:299] Negotiation successful
0924 21:17:10.851460 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260924 21:17:10.851692 26561 catalog_manager.cc:1347] Generated new cluster ID: a5fcd415e773445abeeb4f3dd1b66db5
I20260924 21:17:10.851724 26561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:10.860291 26561 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:10.861063 26561 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:10.866978 26561 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea8958e1c5254ebb935bd1e1655d69ef: Generated new TSK 0
I20260924 21:17:10.867199 26561 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:17:10.895495 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:34383
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:10.975167 26573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:10.975359 26573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:10.975381 26573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:10.976538 26573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:10.976580 26573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:10.976608 26573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:10.978096 26573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:34383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:10.978329 26573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:10.978575 26573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:17:10.983628 26573 server_base.cc:1034] running on GCE node
W20260924 21:17:10.983628 26579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:10.983597 26578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:10.983609 26581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:10.984062 26573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:10.984330 26573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:10.985496 26573 hybrid_clock.cc:648] HybridClock initialized: now 1790284630985481 us; error 26 us; skew 500 ppm
Sep 24 21:17:10 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284630, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:10.987917 26573 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:17:10.988365 26573 webserver.cc:460] Webserver started at http://127.20.115.193:33111/ using document root <none> and password file <none>
I20260924 21:17:10.988590 26573 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:10.988636 26573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:10.988765 26573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:10.989900 26573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "a70a9f7664e5468482ed0858bf46cd9e"
format_stamp: "Formatted at 2026-09-24 21:17:10 on dist-test-slave-cxpd"
server_key: "b6217a64e2bc649339414a5cca5e68e4"
server_key_iv: "42bed3db3bece5f4a2e5292d228938c5"
server_key_version: "encryptionkey@0"
I20260924 21:17:10.990252 26573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "a70a9f7664e5468482ed0858bf46cd9e"
format_stamp: "Formatted at 2026-09-24 21:17:10 on dist-test-slave-cxpd"
server_key: "b6217a64e2bc649339414a5cca5e68e4"
server_key_iv: "42bed3db3bece5f4a2e5292d228938c5"
server_key_version: "encryptionkey@0"
I20260924 21:17:10.991542 26573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:10.992250 26588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:10.992437 26573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:10.992547 26573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "a70a9f7664e5468482ed0858bf46cd9e"
format_stamp: "Formatted at 2026-09-24 21:17:10 on dist-test-slave-cxpd"
server_key: "b6217a64e2bc649339414a5cca5e68e4"
server_key_iv: "42bed3db3bece5f4a2e5292d228938c5"
server_key_version: "encryptionkey@0"
I20260924 21:17:10.992616 26573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:11.018419 26573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:11.020465 26573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:11.020629 26573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:11.020931 26573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:11.021250 26573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:11.021302 26573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:11.021343 26573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:11.021370 26573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:11.027110 26573 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:38969
I20260924 21:17:11.027166 26701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:38969 every 8 connection(s)
I20260924 21:17:11.027498 26573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:17:11.030141 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 26573
I20260924 21:17:11.030267 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:11.030464 20943 external_mini_cluster.cc:1428] Setting key 9c0b504ec8964eb9136b6076e07442ce
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284630, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:11.038156 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:11.028299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:44335 (local address 127.20.115.254:34383)
0924 21:17:11.028341 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:11.028343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:11.028570 (+ 227us) server_negotiation.cc:407] Connection header received
0924 21:17:11.029566 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:11.029567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:11.029576 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:11.029636 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:11.030285 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.031005 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.032022 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.032379 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.032455 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:11.035143 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:11.035145 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:11.035145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:11.035151 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:11.036609 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.037138 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.037139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.037139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.037199 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.037503 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.037503 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.037504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.037609 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:11.037658 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:11.038009 (+ 351us) server_negotiation.cc:299] Negotiation successful
0924 21:17:11.038089 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:17:11.038431 26704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:11.028253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:34383 (local address 127.20.115.193:44335)
0924 21:17:11.028428 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:11.028483 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:11.029476 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:11.029706 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:11.029710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:11.029963 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:11.030146 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:11.030150 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.031097 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.031098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:11.031930 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:11.031932 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.032469 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.032471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:11.032634 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:11.033170 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:11.033181 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:11.035042 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:11.036695 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:11.036702 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:11.036704 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:11.037042 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:11.037290 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:11.037291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:11.037291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:11.037430 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:11.037675 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:11.037681 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:11.037852 (+ 171us) client_negotiation.cc:769] Sending connection context
0924 21:17:11.037974 (+ 122us) client_negotiation.cc:240] Negotiation successful
0924 21:17:11.038169 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
I20260924 21:17:11.039022 26702 heartbeater.cc:347] Connected to a master server at 127.20.115.254:34383
I20260924 21:17:11.039121 26702 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:11.039260 26702 heartbeater.cc:511] Master 127.20.115.254:34383 requested a full tablet report, sending...
I20260924 21:17:11.039776 26511 ts_manager.cc:194] Registered new tserver with Master: a70a9f7664e5468482ed0858bf46cd9e (127.20.115.193:38969)
I20260924 21:17:11.040356 26511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:44335
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:17:11.056233 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:34383
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:11.134080 26707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:11.134270 26707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:11.134292 26707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:11.135437 26707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:11.135478 26707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:11.135504 26707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:11.136986 26707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:34383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:11.137210 26707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:11.137432 26707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:11.142122 26712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:11.142146 26713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:11.142118 26715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:11.142341 26707 server_base.cc:1034] running on GCE node
I20260924 21:17:11.142521 26707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:11.142709 26707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:11.143846 26707 hybrid_clock.cc:648] HybridClock initialized: now 1790284631143827 us; error 34 us; skew 500 ppm
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:11.146298 26707 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:17:11.146783 26707 webserver.cc:460] Webserver started at http://127.20.115.194:37709/ using document root <none> and password file <none>
I20260924 21:17:11.146972 26707 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:11.147012 26707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:11.147105 26707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:11.148211 26707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "cbee0faaa6cb402fa1c5d5a0f08d860e"
format_stamp: "Formatted at 2026-09-24 21:17:11 on dist-test-slave-cxpd"
server_key: "19dc11d8f8ddb2e21aca57e5b577ecd2"
server_key_iv: "7441355385b03677d8062dbdfa39af20"
server_key_version: "encryptionkey@0"
I20260924 21:17:11.148514 26707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "cbee0faaa6cb402fa1c5d5a0f08d860e"
format_stamp: "Formatted at 2026-09-24 21:17:11 on dist-test-slave-cxpd"
server_key: "19dc11d8f8ddb2e21aca57e5b577ecd2"
server_key_iv: "7441355385b03677d8062dbdfa39af20"
server_key_version: "encryptionkey@0"
I20260924 21:17:11.149883 26707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:17:11.150632 26722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:11.150797 26707 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:11.150894 26707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "cbee0faaa6cb402fa1c5d5a0f08d860e"
format_stamp: "Formatted at 2026-09-24 21:17:11 on dist-test-slave-cxpd"
server_key: "19dc11d8f8ddb2e21aca57e5b577ecd2"
server_key_iv: "7441355385b03677d8062dbdfa39af20"
server_key_version: "encryptionkey@0"
I20260924 21:17:11.150971 26707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:11.161315 26707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:11.163072 26707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:11.163230 26707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:11.163455 26707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:11.163746 26707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:11.163800 26707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:11.163841 26707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:11.163867 26707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:11.169675 26707 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:43863
I20260924 21:17:11.169726 26835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:43863 every 8 connection(s)
I20260924 21:17:11.170081 26707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:17:11.170420 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 26707
I20260924 21:17:11.170538 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:11.170717 20943 external_mini_cluster.cc:1428] Setting key 33f63bf2d2f798c830e07dcf9f5dc6f8
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
I20260924 21:17:11.181365 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:11.170901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:33625 (local address 127.20.115.254:34383)
0924 21:17:11.170957 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:11.170959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:11.171167 (+ 208us) server_negotiation.cc:407] Connection header received
0924 21:17:11.172425 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:11.172425 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:11.172435 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:11.172475 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:11.173387 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.173886 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.175671 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.175866 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.175926 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:11.178463 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:11.178464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:11.178465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:11.178472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:11.179910 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.180330 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.180331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.180332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.180390 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.180676 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.180677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.180677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.180842 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:11.180902 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:11.181180 (+ 278us) server_negotiation.cc:299] Negotiation successful
0924 21:17:11.181271 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260924 21:17:11.181571 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:11.170806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:34383 (local address 127.20.115.194:33625)
0924 21:17:11.171023 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:11.171065 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:11.172318 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:11.172565 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:11.172570 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:11.173011 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:11.173289 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:11.173294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.174017 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.174018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:11.175573 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:11.175576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.175963 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.175964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:11.176128 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:11.176683 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:11.176694 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:11.178356 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:11.180003 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:11.180010 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:11.180012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:11.180238 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:11.180456 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:11.180457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:11.180458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:11.180611 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:11.180906 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:11.180913 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:11.181051 (+ 138us) client_negotiation.cc:769] Sending connection context
0924 21:17:11.181161 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 21:17:11.181320 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":86,"threads_started":1}
I20260924 21:17:11.182032 26836 heartbeater.cc:347] Connected to a master server at 127.20.115.254:34383
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
I20260924 21:17:11.182154 26836 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:11.182330 26836 heartbeater.cc:511] Master 127.20.115.254:34383 requested a full tablet report, sending...
I20260924 21:17:11.182767 26511 ts_manager.cc:194] Registered new tserver with Master: cbee0faaa6cb402fa1c5d5a0f08d860e (127.20.115.194:43863)
I20260924 21:17:11.183290 26511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:33625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:17:11.199548 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:34383
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:11.278065 26841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:11.278245 26841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:11.278266 26841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:11.279379 26841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:11.279417 26841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:11.279443 26841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:11.280822 26841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:34383
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:11.280988 26841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:11.281142 26841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:17:11.285953 26841 server_base.cc:1034] running on GCE node
W20260924 21:17:11.285909 26847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:11.286026 26846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:11.285925 26849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:11.286381 26841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:11.286670 26841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:11.287830 26841 hybrid_clock.cc:648] HybridClock initialized: now 1790284631287822 us; error 36 us; skew 500 ppm
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:11.290299 26841 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:17:11.290657 26841 webserver.cc:460] Webserver started at http://127.20.115.195:43663/ using document root <none> and password file <none>
I20260924 21:17:11.290833 26841 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:11.290871 26841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:11.290954 26841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:11.291886 26841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "53d1481c4047419fb976b84b4fff05da"
format_stamp: "Formatted at 2026-09-24 21:17:11 on dist-test-slave-cxpd"
server_key: "991e8118217fbb9c4b8225641890d234"
server_key_iv: "fecf41426ac942e2a66955af26969415"
server_key_version: "encryptionkey@0"
I20260924 21:17:11.292178 26841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "53d1481c4047419fb976b84b4fff05da"
format_stamp: "Formatted at 2026-09-24 21:17:11 on dist-test-slave-cxpd"
server_key: "991e8118217fbb9c4b8225641890d234"
server_key_iv: "fecf41426ac942e2a66955af26969415"
server_key_version: "encryptionkey@0"
I20260924 21:17:11.293460 26841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:11.294113 26856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:11.294281 26841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:11.294353 26841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "53d1481c4047419fb976b84b4fff05da"
format_stamp: "Formatted at 2026-09-24 21:17:11 on dist-test-slave-cxpd"
server_key: "991e8118217fbb9c4b8225641890d234"
server_key_iv: "fecf41426ac942e2a66955af26969415"
server_key_version: "encryptionkey@0"
I20260924 21:17:11.294404 26841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:11.306327 26841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:11.307973 26841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:11.308079 26841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:11.308239 26841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:11.308467 26841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:11.308497 26841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:11.308517 26841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:11.308529 26841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:11.314378 26841 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:36643
I20260924 21:17:11.314445 26969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:36643 every 8 connection(s)
I20260924 21:17:11.314749 26841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:11.323784 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:11.315486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:33679 (local address 127.20.115.254:34383)
0924 21:17:11.315533 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:11.315552 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:11.315733 (+ 181us) server_negotiation.cc:407] Connection header received
0924 21:17:11.316648 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:11.316649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:11.316663 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:11.316710 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:11.317355 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.317857 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.318840 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.319043 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.319132 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:11.321083 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:11.321084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:11.321084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:11.321091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:11.322502 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.322926 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.322927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.322928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.322987 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.323218 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.323219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.323219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.323353 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:11.323437 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:11.323652 (+ 215us) server_negotiation.cc:299] Negotiation successful
0924 21:17:11.323719 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:17:11.323875 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 26841
I20260924 21:17:11.323953 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:11.323997 26972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:11.315444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:34383 (local address 127.20.115.195:33679)
0924 21:17:11.315617 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:11.315656 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:11.316535 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:11.316803 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:11.316807 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:11.317086 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:11.317254 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:11.317258 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.317956 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.317957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:11.318739 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:11.318740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.319154 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.319155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:11.319269 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:11.319672 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:11.319680 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:11.320995 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:11.322569 (+ 1574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:11.322573 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:11.322574 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:11.322853 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:11.323047 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:11.323047 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:11.323047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:11.323135 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:11.323432 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:11.323436 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:11.323541 (+ 105us) client_negotiation.cc:769] Sending connection context
0924 21:17:11.323618 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 21:17:11.323764 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":78,"threads_started":1}
I20260924 21:17:11.324136 20943 external_mini_cluster.cc:1428] Setting key b334ab320b5591b661a80f4e32baf81e
I20260924 21:17:11.324616 26970 heartbeater.cc:347] Connected to a master server at 127.20.115.254:34383
I20260924 21:17:11.324716 26970 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:11.324915 26970 heartbeater.cc:511] Master 127.20.115.254:34383 requested a full tablet report, sending...
I20260924 21:17:11.325304 26511 ts_manager.cc:194] Registered new tserver with Master: 53d1481c4047419fb976b84b4fff05da (127.20.115.195:36643)
I20260924 21:17:11.325630 26511 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:33679
I20260924 21:17:11.335830 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/test-admin.keytab.
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:11.350823 20943 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260924 21:17:11.352514 26977 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:11.351783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55744 (local address 127.20.115.193:38969)
0924 21:17:11.351944 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:11.351948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:11.351957 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:11.351980 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:11.351982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:11.351995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:11.352071 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:11.352423 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55744: BlockingRecv error: recv got EOF from 127.0.0.1:55744 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
W20260924 21:17:11.352591 20943 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5cc' of type FILE: Runtime error: Bad format in credentials cache (filename: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5cc)
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:11.353662 20943 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:11.354569 20943 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
I20260924 21:17:11.362648 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:11.355263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55752 (local address 127.20.115.193:38969)
0924 21:17:11.355319 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:11.355321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:11.355331 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:11.355399 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:11.355400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:11.355418 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:11.355483 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:11.355905 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.356635 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.357439 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.357769 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.357860 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:11.360003 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:11.360010 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:11.360010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:11.360017 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:11.361530 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.361912 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.361915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.361916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.361974 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.362156 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.362157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.362157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.362282 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:11.362475 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:11.362515 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 21:17:11.362579 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260924 21:17:11.364473 26977 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:11.363889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55768 (local address 127.20.115.193:38969)
0924 21:17:11.363923 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:11.363925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:11.364023 (+ 98us) server_negotiation.cc:407] Connection header received
0924 21:17:11.364075 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:11.364075 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:11.364086 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:11.364156 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:11.364426 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55768: BlockingRecv error: recv got EOF from 127.0.0.1:55768 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 21:17:11.364665 20943 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5cc' of type FILE
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:11.365729 20943 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:11.366673 20943 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
I20260924 21:17:11.373510 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:11.367221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55782 (local address 127.20.115.193:38969)
0924 21:17:11.367254 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:11.367256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:11.367353 (+ 97us) server_negotiation.cc:407] Connection header received
0924 21:17:11.367411 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:11.367412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:11.367422 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:11.367494 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:11.367898 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.368371 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.369135 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.369342 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.369417 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:11.371240 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:11.371241 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:11.371242 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:11.371249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:11.372403 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.372738 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.372739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.372740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.372837 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.373043 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.373044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.373045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.373144 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:11.373243 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:11.373348 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 21:17:11.373420 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
W20260924 21:17:11.375010 26977 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:11.374471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.20.115.193:38969)
0924 21:17:11.374506 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:11.374508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:11.374555 (+ 47us) server_negotiation.cc:407] Connection header received
0924 21:17:11.374624 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:11.374625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:11.374634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:11.374687 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:11.374964 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55792: BlockingRecv error: recv got EOF from 127.0.0.1:55792 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:17:11.375095 20943 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790284606872664-20943-0/krb5kdc/krb5cc' of type FILE
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:11.376005 20943 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:11.376871 20943 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26466](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.193@KRBTEST.COM
I20260924 21:17:11.383169 26977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:11.377315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55804 (local address 127.20.115.193:38969)
0924 21:17:11.377366 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:11.377368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:11.377404 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:17:11.377460 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:11.377461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:11.377468 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:11.377513 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:11.377820 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.378248 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.378851 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:11.379023 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:11.379071 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:11.381002 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:11.381003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:11.381003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:11.381008 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:11.382311 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.382599 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.382599 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.382600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.382641 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:11.382775 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:11.382775 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:11.382775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:11.382847 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:11.382906 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:11.382991 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 21:17:11.383039 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:17:11.383808 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 26573
I20260924 21:17:11.388528 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 26707
I20260924 21:17:11.392584 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 26841
I20260924 21:17:11.396684 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 26479
2026-09-24T21:17:11Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1810 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26990](info): set up 2 sockets
Sep 24 21:17:11 dist-test-slave-cxpd krb5kdc[26990](info): commencing operation
krb5kdc: starting...
W20260924 21:17:12.449005 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:12 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.254.keytab.
Entry for principal oryx/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.254.keytab.
I20260924 21:17:12.565124 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:40423
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.254.keytab
--principal=oryx/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:40423
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:12.642702 27003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:12.642911 27003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:12.642933 27003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:12.644109 27003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:12.644153 27003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:12.644167 27003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:12.644181 27003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:12.644192 27003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:12.645669 27003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:40423
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.254.keytab
--principal=oryx/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:40423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:12.645931 27003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:12.646175 27003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:12.651163 27011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:12.651154 27009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:12.651340 27003 server_base.cc:1034] running on GCE node
W20260924 21:17:12.651165 27008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:12.651620 27003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:12.651860 27003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:12.653069 27003 hybrid_clock.cc:648] HybridClock initialized: now 1790284632653046 us; error 38 us; skew 500 ppm
Sep 24 21:17:12 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284632, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:12.660274 27003 init.cc:377] Logged in from keytab as oryx/127.20.115.254@KRBTEST.COM (short username oryx)
I20260924 21:17:12.660825 27003 webserver.cc:460] Webserver started at http://127.20.115.254:46345/ using document root <none> and password file <none>
I20260924 21:17:12.661060 27003 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:12.661104 27003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:12.661254 27003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:12.662333 27003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "da840c06cc98432690b005cdba3e5e4d"
format_stamp: "Formatted at 2026-09-24 21:17:12 on dist-test-slave-cxpd"
server_key: "c5ea54055b4bce3e4aaa221c1eba9ded"
server_key_iv: "02cc1249c6b81122128d7334976fb6f1"
server_key_version: "encryptionkey@0"
I20260924 21:17:12.662707 27003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "da840c06cc98432690b005cdba3e5e4d"
format_stamp: "Formatted at 2026-09-24 21:17:12 on dist-test-slave-cxpd"
server_key: "c5ea54055b4bce3e4aaa221c1eba9ded"
server_key_iv: "02cc1249c6b81122128d7334976fb6f1"
server_key_version: "encryptionkey@0"
I20260924 21:17:12.664091 27003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:12.664944 27018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:12.665104 27003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:12.665192 27003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "da840c06cc98432690b005cdba3e5e4d"
format_stamp: "Formatted at 2026-09-24 21:17:12 on dist-test-slave-cxpd"
server_key: "c5ea54055b4bce3e4aaa221c1eba9ded"
server_key_iv: "02cc1249c6b81122128d7334976fb6f1"
server_key_version: "encryptionkey@0"
I20260924 21:17:12.665246 27003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:12.691463 27003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:12.693764 27003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:12.693877 27003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:12.697542 27003 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:40423
I20260924 21:17:12.697602 27070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:40423 every 8 connection(s)
I20260924 21:17:12.697926 27003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:12.698871 27071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:12.699398 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 27003
I20260924 21:17:12.699491 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:12.699703 20943 external_mini_cluster.cc:1428] Setting key efc07e2f7161e414608008363490b7c7
I20260924 21:17:12.701807 27071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: Bootstrap starting.
I20260924 21:17:12.702378 27071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:12.702663 27071 log.cc:824] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:12.703392 27071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: No bootstrap required, opened a new log
I20260924 21:17:12.705804 27071 raft_consensus.cc:348] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } }
I20260924 21:17:12.705955 27071 raft_consensus.cc:374] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:12.705979 27071 raft_consensus.cc:729] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da840c06cc98432690b005cdba3e5e4d, State: Initialized, Role: FOLLOWER
I20260924 21:17:12.706106 27071 consensus_queue.cc:260] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } }
I20260924 21:17:12.706200 27071 raft_consensus.cc:388] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:12.706243 27071 raft_consensus.cc:482] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:12.706292 27071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 0 FOLLOWER]: Advancing to term 1
Sep 24 21:17:12 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:12.706851 27071 raft_consensus.cc:504] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } }
I20260924 21:17:12.706935 27071 leader_election.cc:302] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da840c06cc98432690b005cdba3e5e4d; no voters:
I20260924 21:17:12.707109 27071 leader_election.cc:288] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:12.707140 27076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:12.707304 27076 raft_consensus.cc:686] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 1 LEADER]: Becoming Leader. State: Replica: da840c06cc98432690b005cdba3e5e4d, State: Running, Role: LEADER
I20260924 21:17:12.707358 27071 sys_catalog.cc:564] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:12.707406 27076 consensus_queue.cc:237] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } }
I20260924 21:17:12.707720 27077 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader da840c06cc98432690b005cdba3e5e4d. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } }
I20260924 21:17:12.707813 27077 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:12.708027 27078 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } }
I20260924 21:17:12.708108 27078 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:12.708225 27083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:12.708995 27083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:12.710966 27083 catalog_manager.cc:1347] Generated new cluster ID: 8353c0d9a88f4474ba76971f10fd583e
I20260924 21:17:12.711017 27083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:12.711714 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:12.700265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45312 (local address 127.20.115.254:40423)
0924 21:17:12.700429 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:12.700436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:12.700459 (+ 23us) server_negotiation.cc:407] Connection header received
0924 21:17:12.700897 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:12.700903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:12.700948 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:12.701056 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:12.701819 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:12.702767 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:12.703734 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:12.704129 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:12.704210 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:12.707054 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:12.707059 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:12.707059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:12.707072 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:12.710018 (+ 2946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:12.710420 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:12.710423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:12.710423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:12.710521 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:12.710718 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:12.710720 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:12.710720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:12.711045 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:12.711172 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:12.711394 (+ 222us) server_negotiation.cc:299] Negotiation successful
0924 21:17:12.711534 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260924 21:17:12.732820 27083 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:12.733541 27083 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:12.742161 27083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: Generated new TSK 0
I20260924 21:17:12.742348 27083 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab.
Entry for principal oryx/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab.
I20260924 21:17:12.768345 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40423
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab
--principal=oryx/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:12.848537 27097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:12.848729 27097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:12.848753 27097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:12.849922 27097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:12.849965 27097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:12.849992 27097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:12.851455 27097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab
--principal=oryx/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:12.851688 27097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:12.851927 27097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:12.856930 27102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:12.856931 27105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:12.857026 27103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:12.857084 27097 server_base.cc:1034] running on GCE node
I20260924 21:17:12.857334 27097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:12.857583 27097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:12.858774 27097 hybrid_clock.cc:648] HybridClock initialized: now 1790284632858768 us; error 50 us; skew 500 ppm
Sep 24 21:17:12 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284632, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:12.861315 27097 init.cc:377] Logged in from keytab as oryx/127.20.115.193@KRBTEST.COM (short username oryx)
I20260924 21:17:12.861732 27097 webserver.cc:460] Webserver started at http://127.20.115.193:35211/ using document root <none> and password file <none>
I20260924 21:17:12.861963 27097 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:12.862031 27097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:12.862175 27097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:12.863189 27097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "7d4cf1aa4744407db2cb3454411b7c00"
format_stamp: "Formatted at 2026-09-24 21:17:12 on dist-test-slave-cxpd"
server_key: "bfc03db763a5b94a154057c00964d75f"
server_key_iv: "ed9e0e20a81eae4c0e12274d165c8680"
server_key_version: "encryptionkey@0"
I20260924 21:17:12.863544 27097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "7d4cf1aa4744407db2cb3454411b7c00"
format_stamp: "Formatted at 2026-09-24 21:17:12 on dist-test-slave-cxpd"
server_key: "bfc03db763a5b94a154057c00964d75f"
server_key_iv: "ed9e0e20a81eae4c0e12274d165c8680"
server_key_version: "encryptionkey@0"
I20260924 21:17:12.864890 27097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:17:12.865670 27112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:12.865891 27097 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:12.865979 27097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "7d4cf1aa4744407db2cb3454411b7c00"
format_stamp: "Formatted at 2026-09-24 21:17:12 on dist-test-slave-cxpd"
server_key: "bfc03db763a5b94a154057c00964d75f"
server_key_iv: "ed9e0e20a81eae4c0e12274d165c8680"
server_key_version: "encryptionkey@0"
I20260924 21:17:12.866062 27097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:12.887673 27097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:12.889608 27097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:12.889765 27097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:12.889988 27097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:12.890273 27097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:12.890323 27097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:12.890362 27097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:12.890390 27097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:12.895980 27097 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:42089
I20260924 21:17:12.896052 27225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:42089 every 8 connection(s)
I20260924 21:17:12.896356 27097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:17:12 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284632, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:12.903184 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 27097
I20260924 21:17:12.903282 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:12.903520 20943 external_mini_cluster.cc:1428] Setting key 95ea179d498f93603f6a7dea234efd75
I20260924 21:17:12.905289 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:12.897103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:36097 (local address 127.20.115.254:40423)
0924 21:17:12.897171 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:12.897173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:12.897335 (+ 162us) server_negotiation.cc:407] Connection header received
0924 21:17:12.897803 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:12.897803 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:12.897812 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:12.897863 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:12.898759 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:12.899256 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:12.900171 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:12.900363 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:12.900414 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:12.902345 (+ 1931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:12.902347 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:12.902347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:12.902354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:12.903852 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:12.904306 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:12.904307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:12.904307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:12.904359 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:12.904635 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:12.904635 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:12.904636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:12.904733 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:12.904826 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:12.905153 (+ 327us) server_negotiation.cc:299] Negotiation successful
0924 21:17:12.905226 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 21:17:12.905623 27228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:12.897052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.193:36097)
0924 21:17:12.897212 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:12.897251 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:12.897723 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:12.897920 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:12.897923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:12.898501 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:12.898684 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:12.898689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:12.899334 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:12.899334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:12.900096 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:12.900098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:12.900431 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:12.900431 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:12.900523 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:12.900964 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:12.900971 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:12.902261 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:12.903925 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:12.903935 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:12.903937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:12.904230 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:12.904425 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:12.904426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:12.904426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:12.904570 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:12.904819 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:12.904825 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:12.905017 (+ 192us) client_negotiation.cc:769] Sending connection context
0924 21:17:12.905131 (+ 114us) client_negotiation.cc:240] Negotiation successful
0924 21:17:12.905301 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260924 21:17:12.906186 27226 heartbeater.cc:347] Connected to a master server at 127.20.115.254:40423
I20260924 21:17:12.906287 27226 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:12.906457 27226 heartbeater.cc:511] Master 127.20.115.254:40423 requested a full tablet report, sending...
I20260924 21:17:12.906980 27035 ts_manager.cc:194] Registered new tserver with Master: 7d4cf1aa4744407db2cb3454411b7c00 (127.20.115.193:42089)
I20260924 21:17:12.907670 27035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:36097
WARNING: no policy specified for oryx/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab.
Entry for principal oryx/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab.
I20260924 21:17:12.928042 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40423
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab
--principal=oryx/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:13.006444 27231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:13.006635 27231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:13.006659 27231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:13.007838 27231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:13.007877 27231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:13.007903 27231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:13.009346 27231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab
--principal=oryx/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:13.009575 27231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:13.009804 27231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:13.014575 27236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.014560 27237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.014551 27239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.014786 27231 server_base.cc:1034] running on GCE node
I20260924 21:17:13.015053 27231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:13.015296 27231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:13.016482 27231 hybrid_clock.cc:648] HybridClock initialized: now 1790284633016466 us; error 32 us; skew 500 ppm
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:13.019030 27231 init.cc:377] Logged in from keytab as oryx/127.20.115.194@KRBTEST.COM (short username oryx)
I20260924 21:17:13.019477 27231 webserver.cc:460] Webserver started at http://127.20.115.194:45359/ using document root <none> and password file <none>
I20260924 21:17:13.019690 27231 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.019755 27231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.019903 27231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:13.020952 27231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "0edc40f3b1034ae9806e1cb201c1c793"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "36c0e0603dc40cf21f07f1d0616f4d1c"
server_key_iv: "045e25700e8b9032f40caeb3b258c4c6"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.021291 27231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "0edc40f3b1034ae9806e1cb201c1c793"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "36c0e0603dc40cf21f07f1d0616f4d1c"
server_key_iv: "045e25700e8b9032f40caeb3b258c4c6"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.022526 27231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:13.023198 27246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.023365 27231 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:13.023458 27231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "0edc40f3b1034ae9806e1cb201c1c793"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "36c0e0603dc40cf21f07f1d0616f4d1c"
server_key_iv: "045e25700e8b9032f40caeb3b258c4c6"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.023537 27231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.046952 27231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:13.049232 27231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:13.049387 27231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:13.049618 27231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:13.049912 27231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:13.049973 27231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.050014 27231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:13.050041 27231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.056162 27231 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:35431
I20260924 21:17:13.056207 27359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:35431 every 8 connection(s)
I20260924 21:17:13.056535 27231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:13.062517 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 27231
I20260924 21:17:13.062644 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:13.062829 20943 external_mini_cluster.cc:1428] Setting key 1ceaca4a17ee26d8352ddbfa4b456736
I20260924 21:17:13.065881 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.057319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:46077 (local address 127.20.115.254:40423)
0924 21:17:13.057414 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.057416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.057578 (+ 162us) server_negotiation.cc:407] Connection header received
0924 21:17:13.058060 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.058061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.058071 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.058127 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.059006 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.059520 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.060483 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.060670 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.060732 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.062965 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.062967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.062968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.062975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.064417 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.064935 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.064936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.064937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.065010 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.065247 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.065248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.065249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.065386 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.065465 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.065683 (+ 218us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.065766 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260924 21:17:13.065896 27362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.057274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.194:46077)
0924 21:17:13.057447 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.057482 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.057963 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.058179 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.058182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.058723 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.058903 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.058907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.059603 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.059604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.060383 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.060385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.060749 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.060750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.060947 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.061445 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.061453 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.062883 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.064495 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.064501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.064503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.064840 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.065087 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.065087 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.065088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.065182 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.065455 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.065459 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.065561 (+ 102us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.065638 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.065748 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
I20260924 21:17:13.066435 27360 heartbeater.cc:347] Connected to a master server at 127.20.115.254:40423
I20260924 21:17:13.066524 27360 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:13.066702 27360 heartbeater.cc:511] Master 127.20.115.254:40423 requested a full tablet report, sending...
I20260924 21:17:13.067071 27035 ts_manager.cc:194] Registered new tserver with Master: 0edc40f3b1034ae9806e1cb201c1c793 (127.20.115.194:35431)
I20260924 21:17:13.067559 27035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:46077
WARNING: no policy specified for oryx/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab.
Entry for principal oryx/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab.
I20260924 21:17:13.085326 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40423
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab
--principal=oryx/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:13.163856 27365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:13.164067 27365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:13.164090 27365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:13.165400 27365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:13.165446 27365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:13.165473 27365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:13.166941 27365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab
--principal=oryx/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:13.167189 27365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:13.167441 27365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:13.172744 27371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.172751 27373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.172809 27370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.173049 27365 server_base.cc:1034] running on GCE node
I20260924 21:17:13.173272 27365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:13.173485 27365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:13.174645 27365 hybrid_clock.cc:648] HybridClock initialized: now 1790284633174631 us; error 25 us; skew 500 ppm
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:13.177300 27365 init.cc:377] Logged in from keytab as oryx/127.20.115.195@KRBTEST.COM (short username oryx)
I20260924 21:17:13.177742 27365 webserver.cc:460] Webserver started at http://127.20.115.195:33761/ using document root <none> and password file <none>
I20260924 21:17:13.178004 27365 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.178076 27365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.178224 27365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:13.179394 27365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "1a7da08f0aa6420597f34f1333393d80"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "c84af5aac66b517808d2c9ef812d4b59"
server_key_iv: "ce231d35cbb03ffd35ba8468153fc5bf"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.179780 27365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "1a7da08f0aa6420597f34f1333393d80"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "c84af5aac66b517808d2c9ef812d4b59"
server_key_iv: "ce231d35cbb03ffd35ba8468153fc5bf"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.181187 27365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:17:13.181927 27380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.182096 27365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:13.182200 27365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "1a7da08f0aa6420597f34f1333393d80"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "c84af5aac66b517808d2c9ef812d4b59"
server_key_iv: "ce231d35cbb03ffd35ba8468153fc5bf"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.182273 27365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.208824 27365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:13.211035 27365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:13.211191 27365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:13.211417 27365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:13.211723 27365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:13.211773 27365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.211808 27365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:13.211839 27365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.217813 27365 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:45431
I20260924 21:17:13.217852 27493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:45431 every 8 connection(s)
I20260924 21:17:13.218170 27365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:17:13.219579 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 27365
I20260924 21:17:13.219695 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:13.219939 20943 external_mini_cluster.cc:1428] Setting key e260df80ec417b5222f8e3c5ab076173
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:13.227157 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.218937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:42917 (local address 127.20.115.254:40423)
0924 21:17:13.218983 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.218985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.219111 (+ 126us) server_negotiation.cc:407] Connection header received
0924 21:17:13.219648 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.219650 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.219664 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.219764 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.221076 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.221566 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.222526 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.222709 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.222765 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.224824 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.224825 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.224826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.224838 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.225989 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.226399 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.226399 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.226400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.226451 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.226661 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.226661 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.226661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.226757 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.226806 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.227037 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.227095 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:17:13.227443 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.218841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:42917)
0924 21:17:13.219000 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.219041 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.219548 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.219904 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.219910 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.220732 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.220997 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.221002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.221649 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.221649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.222449 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.222451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.222806 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.222807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.222913 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.223349 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.223357 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.224710 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.226116 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.226122 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.226124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.226335 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.226513 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.226514 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.226514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.226607 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.226812 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.226816 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.226922 (+ 106us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.227010 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.227174 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":94,"threads_started":1}
I20260924 21:17:13.227905 27494 heartbeater.cc:347] Connected to a master server at 127.20.115.254:40423
I20260924 21:17:13.228024 27494 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:13.228192 27494 heartbeater.cc:511] Master 127.20.115.254:40423 requested a full tablet report, sending...
I20260924 21:17:13.228576 27035 ts_manager.cc:194] Registered new tserver with Master: 1a7da08f0aa6420597f34f1333393d80 (127.20.115.195:45431)
I20260924 21:17:13.229045 27035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:42917
I20260924 21:17:13.231467 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.115.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.253.keytab.
Entry for principal oryx/127.20.115.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.253.keytab.
I20260924 21:17:13.256173 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.253:41823
--webserver_interface=127.20.115.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.253.keytab
--principal=oryx/127.20.115.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:40423,127.20.115.253:41823
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:13.335500 27499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:13.335724 27499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:13.335763 27499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:13.336925 27499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:13.336993 27499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:13.337018 27499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:13.337044 27499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:13.337074 27499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:13.338526 27499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:40423,127.20.115.253:41823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.253.keytab
--principal=oryx/127.20.115.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.253:41823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.115.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
W20260924 21:17:13.338657 27499 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.115.254:40423,127.20.115.253:41823'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260924 21:17:13.338814 27499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:13.339023 27499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:13.343694 27507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.343688 27505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.343686 27504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.343901 27499 server_base.cc:1034] running on GCE node
I20260924 21:17:13.344197 27499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:13.344478 27499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:13.345635 27499 hybrid_clock.cc:648] HybridClock initialized: now 1790284633345627 us; error 45 us; skew 500 ppm
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:13.348024 27499 init.cc:377] Logged in from keytab as oryx/127.20.115.253@KRBTEST.COM (short username oryx)
I20260924 21:17:13.348489 27499 webserver.cc:460] Webserver started at http://127.20.115.253:35841/ using document root <none> and password file <none>
I20260924 21:17:13.348706 27499 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.348773 27499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.348953 27499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:13.349963 27499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data/instance:
uuid: "b85dbe21632b4af8951f35fa0dac492d"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "e2d59e43209d8c696319ab6d201e8457"
server_key_iv: "097a16590905e8c39fcac2320cd86ca3"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.350320 27499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal/instance:
uuid: "b85dbe21632b4af8951f35fa0dac492d"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "e2d59e43209d8c696319ab6d201e8457"
server_key_iv: "097a16590905e8c39fcac2320cd86ca3"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.351621 27499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:13.352327 27514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.352494 27499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:13.352595 27499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal
uuid: "b85dbe21632b4af8951f35fa0dac492d"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "e2d59e43209d8c696319ab6d201e8457"
server_key_iv: "097a16590905e8c39fcac2320cd86ca3"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.352675 27499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.366304 27499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:13.368100 27499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:13.368253 27499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:13.372362 27499 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.253:41823
I20260924 21:17:13.372391 27566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.253:41823 every 8 connection(s)
I20260924 21:17:13.372713 27499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
I20260924 21:17:13.373623 27567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:13.375152 27567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.253@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:13.380972 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 27499
I20260924 21:17:13.383991 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.375961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45314 (local address 127.20.115.254:40423)
0924 21:17:13.376010 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.376013 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.376258 (+ 245us) server_negotiation.cc:407] Connection header received
0924 21:17:13.376670 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.376671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.376681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.376732 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.377319 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.377794 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.378809 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.379000 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.379054 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.381165 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.381165 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.381166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.381172 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.382612 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.383024 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.383025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.383026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.383087 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.383333 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.383334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.383334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.383485 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.383560 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.383823 (+ 263us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.383903 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:17:13.384163 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.375961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45314)
0924 21:17:13.376142 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.376174 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.376588 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.376817 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.376820 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.377061 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.377245 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.377249 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.377868 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.377868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.378726 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.378728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.379079 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.379080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.379177 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.379634 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.379641 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.381082 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.382717 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.382723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.382725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.382945 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.383156 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.383157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.383157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.383263 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.383563 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.383569 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.383695 (+ 126us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.383796 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.383947 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
I20260924 21:17:13.384737 27567 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.115.253@KRBTEST.COM
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.253@KRBTEST.COM for oryx/127.20.115.253@KRBTEST.COM
I20260924 21:17:13.390215 27571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.381460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41980 (local address 127.20.115.253:41823)
0924 21:17:13.381625 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.381630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.381711 (+ 81us) server_negotiation.cc:407] Connection header received
0924 21:17:13.381758 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.381760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.381773 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.381848 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.382425 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.383075 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.383798 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.384168 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.384234 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.387228 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.387236 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.387236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.387243 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.388981 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.389377 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.389380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.389380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.389447 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.389657 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.389658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.389658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.389797 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.389996 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.390039 (+ 43us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.390112 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":66,"threads_started":1}
I20260924 21:17:13.392766 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.384990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.0.0.1:41982)
0924 21:17:13.385042 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.385053 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.385092 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.385350 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.385350 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.385528 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.385579 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.385580 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.386415 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.386416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.387176 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.387179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.387691 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.387692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.387780 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.388173 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.388178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.389568 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.391816 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.391816 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.391817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.391974 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.392178 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.392178 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.392179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.392232 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.392478 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.392478 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.392513 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.392575 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.392663 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 21:17:13.392874 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.385041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41982 (local address 127.20.115.253:41823)
0924 21:17:13.385189 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.385191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.385199 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:13.385219 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.385220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.385231 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.385321 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.385653 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.386337 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.387248 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.387576 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.387668 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.389658 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.389659 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.389660 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.389666 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.391715 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.392054 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.392055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.392056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.392106 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.392298 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.392298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.392299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.392412 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.392487 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.392594 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.392711 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260924 21:17:13.393908 27567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Bootstrap starting.
I20260924 21:17:13.394430 27567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:13.394644 27567 log.cc:824] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:13.395494 27567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: No bootstrap required, opened a new log
I20260924 21:17:13.397025 27567 raft_consensus.cc:348] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } }
I20260924 21:17:13.397177 27567 raft_consensus.cc:374] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:13.397221 27567 raft_consensus.cc:729] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b85dbe21632b4af8951f35fa0dac492d, State: Initialized, Role: FOLLOWER
I20260924 21:17:13.397336 27567 consensus_queue.cc:260] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } }
I20260924 21:17:13.397631 27567 sys_catalog.cc:564] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:13.397593 27574 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } } }
I20260924 21:17:13.397748 27574 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:17:13.399375 27585 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:17:13.399436 27585 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:17:13.401142 27499 master_runner.cc:428] Detected that this master b85dbe21632b4af8951f35fa0dac492d is joining an existing cluster
I20260924 21:17:13.401183 27499 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.115.253:41823
I20260924 21:17:13.408730 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.402142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45328 (local address 127.20.115.254:40423)
0924 21:17:13.402186 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.402188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.402296 (+ 108us) server_negotiation.cc:407] Connection header received
0924 21:17:13.402422 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.402422 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.402433 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.402495 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.402859 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.403655 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.404423 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.404619 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.404703 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.405726 (+ 1023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.405727 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.405728 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.405734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.407424 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.407910 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.407911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.407912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.408013 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.408216 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.408217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.408218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.408348 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.408416 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.408567 (+ 151us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.408644 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:17:13.408936 27591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.402109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45328)
0924 21:17:13.402238 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.402248 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.402341 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.402496 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.402497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.402724 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.402783 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.402785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.403750 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.403751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.404335 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.404336 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.404705 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.404706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.404767 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.405227 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.405232 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.405643 (+ 411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.407533 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.407534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.407535 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.407821 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.408066 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.408067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.408067 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.408125 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.408407 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.408408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.408454 (+ 46us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.408541 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.408640 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":30,"thread_start_us":76,"threads_started":1}
I20260924 21:17:13.409515 27592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.402147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.0.0.1:41986)
0924 21:17:13.402311 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.402321 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.402378 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.402633 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.402634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.402835 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.402910 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.402912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.403759 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.403760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.404642 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.404644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.405209 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.405209 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.405265 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.405658 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.405663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.406037 (+ 374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.407924 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.407925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.407925 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.408168 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.408361 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.408361 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.408362 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.408428 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.408731 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.408732 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.408765 (+ 33us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.409312 (+ 547us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.409416 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":3,"thread_start_us":71,"threads_started":1}
I20260924 21:17:13.409597 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.402292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.20.115.253:41823)
0924 21:17:13.402321 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.402323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.402365 (+ 42us) server_negotiation.cc:407] Connection header received
0924 21:17:13.402539 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.402540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.402549 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.402599 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.402992 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.403680 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.404753 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.405098 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.405141 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.406143 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.406144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.406144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.406151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.407837 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.408256 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.408257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.408257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.408299 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.408500 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.408501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.408502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.408634 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.408677 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.409353 (+ 676us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.409487 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":17}
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284632, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.254@KRBTEST.COM for oryx/127.20.115.253@KRBTEST.COM
I20260924 21:17:13.418291 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.411585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.20.115.253:41823)
0924 21:17:13.411623 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.411624 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.411635 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:13.411723 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.411723 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.411732 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.411787 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.412179 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.412618 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.413458 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.413650 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.413733 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.415765 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.415766 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.415766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.415773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.417153 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.417518 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.417519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.417519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.417571 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.417774 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.417775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.417775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.417878 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.417950 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.418140 (+ 190us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.418211 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:17:13.418305 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.411387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.0.0.1:41996)
0924 21:17:13.411527 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.411591 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.411659 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.411792 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.411795 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.412057 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.412098 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.412099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.412698 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.412699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.413350 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.413352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.413737 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.413739 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.413812 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.414244 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.414248 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.415677 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.417229 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.417236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.417238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.417431 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.417645 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.417646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.417646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.417705 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.417944 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.417958 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.418006 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.418103 (+ 97us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.418215 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260924 21:17:13.418972 27035 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.115.253:41823: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d"
member_type: NON_VOTER
last_known_addr {
host: "127.20.115.253"
port: 41823
}
attrs {
promote: true
}
}
dest_uuid: "da840c06cc98432690b005cdba3e5e4d"
cas_config_opid_index: -1
I20260924 21:17:13.419267 27035 consensus_queue.cc:237] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } }
I20260924 21:17:13.419682 27595 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b85dbe21632b4af8951f35fa0dac492d (127.20.115.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } } }
I20260924 21:17:13.419958 27541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:13.420734 27595 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.115.253:41823
I20260924 21:17:13.420835 27541 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Refusing update from remote peer da840c06cc98432690b005cdba3e5e4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260924 21:17:13.420967 27499 master.cc:537] Master@127.20.115.253:41823 shutting down...
I20260924 21:17:13.421139 27595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:13.421185 27594 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } } }
I20260924 21:17:13.421288 27594 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
W20260924 21:17:13.421919 27021 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260924 21:17:13.423336 27021 consensus_peers.cc:487] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d -> Peer b85dbe21632b4af8951f35fa0dac492d (127.20.115.253:41823): Couldn't send request to peer b85dbe21632b4af8951f35fa0dac492d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 21:17:13.423993 27499 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:13.424079 27499 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:13.424125 27499 tablet_replica.cc:330] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: stopping tablet replica
I20260924 21:17:13.436676 27499 master.cc:552] Master@127.20.115.253:41823 shutdown complete.
I20260924 21:17:13.436743 27499 master_runner.cc:305] Clearing existing system tablet
I20260924 21:17:13.436909 27499 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.436971 27499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.437574 27499 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:13.438064 27600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.438189 27499 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.438249 27499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.438287 27499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal
uuid: "b85dbe21632b4af8951f35fa0dac492d"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "e2d59e43209d8c696319ab6d201e8457"
server_key_iv: "097a16590905e8c39fcac2320cd86ca3"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.438853 27499 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:13.440312 27499 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 21:17:13.440367 27499 log.cc:1196] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260924 21:17:13.440562 27499 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Deleting consensus metadata
I20260924 21:17:13.440691 27499 master_runner.cc:315] Copying system tablet from 127.20.115.254:40423
I20260924 21:17:13.441288 27499 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: tablet copy: Beginning tablet copy session from remote peer at address 127.20.115.254:40423
I20260924 21:17:13.447288 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.441622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45340 (local address 127.20.115.254:40423)
0924 21:17:13.441664 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.441667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.441706 (+ 39us) server_negotiation.cc:407] Connection header received
0924 21:17:13.441780 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.441781 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.441789 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.441834 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.442169 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.442643 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.443678 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.443850 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.443906 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.444865 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.444866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.444866 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.444874 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.446252 (+ 1378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.446623 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.446623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.446624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.446672 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.446857 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.446857 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.446857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.446967 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.447016 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.447179 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.447232 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:17:13.447358 27606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.441532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45340)
0924 21:17:13.441648 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.441659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.441722 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.441841 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.441841 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.442034 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.442101 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.442103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.442714 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.442715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.443597 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.443600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.443918 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.443918 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.443987 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.444395 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.444400 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.444761 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.446332 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.446333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.446333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.446549 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.446733 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.446734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.446735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.446797 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.447026 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.447027 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.447072 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.447160 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.447253 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":80,"threads_started":1}
I20260924 21:17:13.447929 27055 tablet_copy_service.cc:140] P da840c06cc98432690b005cdba3e5e4d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b85dbe21632b4af8951f35fa0dac492d ({username='oryx', principal='oryx/127.20.115.253@KRBTEST.COM'} at 127.0.0.1:45340)
I20260924 21:17:13.448029 27055 tablet_copy_service.cc:161] P da840c06cc98432690b005cdba3e5e4d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b85dbe21632b4af8951f35fa0dac492d at {username='oryx', principal='oryx/127.20.115.253@KRBTEST.COM'} at 127.0.0.1:45340: session id = b85dbe21632b4af8951f35fa0dac492d-00000000000000000000000000000000
I20260924 21:17:13.448896 27055 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 21:17:13.449466 27499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 21:17:13.450277 27499 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: tablet copy: Starting download of 0 data blocks...
I20260924 21:17:13.450439 27499 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: tablet copy: Starting download of 1 WAL segments...
I20260924 21:17:13.451319 27499 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 21:17:13.452011 27055 tablet_copy_service.cc:342] P da840c06cc98432690b005cdba3e5e4d: Request end of tablet copy session b85dbe21632b4af8951f35fa0dac492d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.115.253@KRBTEST.COM'} at 127.0.0.1:45340
I20260924 21:17:13.452072 27055 tablet_copy_service.cc:433] P da840c06cc98432690b005cdba3e5e4d: ending tablet copy session b85dbe21632b4af8951f35fa0dac492d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b85dbe21632b4af8951f35fa0dac492d
I20260924 21:17:13.454459 27499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:13.455603 27607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.455667 27608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.455801 27499 server_base.cc:1034] running on GCE node
W20260924 21:17:13.460915 27610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.461079 27499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:13.461230 27499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:13.462352 27499 hybrid_clock.cc:648] HybridClock initialized: now 1790284633462343 us; error 26 us; skew 500 ppm
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:13.463673 27499 init.cc:377] Logged in from keytab as oryx/127.20.115.253@KRBTEST.COM (short username oryx)
I20260924 21:17:13.463922 27499 webserver.cc:460] Webserver started at http://127.20.115.253:36737/ using document root <none> and password file <none>
I20260924 21:17:13.464056 27499 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.464108 27499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.464680 27499 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:13.465260 27617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.465389 27499 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.465436 27499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal
uuid: "b85dbe21632b4af8951f35fa0dac492d"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "e2d59e43209d8c696319ab6d201e8457"
server_key_iv: "097a16590905e8c39fcac2320cd86ca3"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.465489 27499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.483130 27499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:13.484851 27499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:13.487922 27499 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.253:41823
I20260924 21:17:13.487957 27669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.253:41823 every 8 connection(s)
I20260924 21:17:13.488248 27499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
I20260924 21:17:13.489118 27670 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:17:13.489614 27670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Bootstrap starting.
I20260924 21:17:13.492460 27670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:17:13.492712 27670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Bootstrap complete.
I20260924 21:17:13.492946 27670 raft_consensus.cc:348] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } }
I20260924 21:17:13.493037 27670 raft_consensus.cc:729] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b85dbe21632b4af8951f35fa0dac492d, State: Initialized, Role: LEARNER
I20260924 21:17:13.493075 27670 consensus_queue.cc:260] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } }
I20260924 21:17:13.493237 27670 sys_catalog.cc:564] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:13.493237 27674 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } } }
I20260924 21:17:13.493417 27674 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: This master's current role is: LEARNER
I20260924 21:17:13.495005 27685 catalog_manager.cc:1259] Loaded cluster ID: 8353c0d9a88f4474ba76971f10fd583e
I20260924 21:17:13.495085 27685 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: loading cluster ID for follower catalog manager: success
I20260924 21:17:13.496271 27685 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: acquiring CA information for follower catalog manager: success
I20260924 21:17:13.496666 27685 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:17:13.498814 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.491658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42026 (local address 127.20.115.253:41823)
0924 21:17:13.491826 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.491828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.491833 (+ 5us) server_negotiation.cc:407] Connection header received
0924 21:17:13.491851 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.491851 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.491858 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.491928 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.492448 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.493383 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.494278 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.494674 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.495203 (+ 529us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.496078 (+ 875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.496079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.496080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.496085 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.497691 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.498051 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.498052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.498053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.498113 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.498310 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.498311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.498311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.498419 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.498529 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.498634 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.498721 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for oryx/127.20.115.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.252.keytab.
Entry for principal oryx/127.20.115.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.252.keytab.
I20260924 21:17:13.523774 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.252:32805
--webserver_interface=127.20.115.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.252.keytab
--principal=oryx/127.20.115.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:40423,127.20.115.253:41823,127.20.115.252:32805
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:13.603653 27688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:13.603852 27688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:13.603873 27688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:13.605051 27688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:13.605090 27688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:13.605105 27688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:13.605118 27688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:13.605130 27688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:13.606629 27688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:40423,127.20.115.253:41823,127.20.115.252:32805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.252.keytab
--principal=oryx/127.20.115.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.252:32805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.115.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:13.606877 27688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:13.607188 27688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:13.612022 27694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.612152 27688 server_base.cc:1034] running on GCE node
W20260924 21:17:13.612025 27696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.612306 27693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.612569 27688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:13.612901 27688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:13.614084 27688 hybrid_clock.cc:648] HybridClock initialized: now 1790284633614061 us; error 35 us; skew 500 ppm
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:13.616446 27688 init.cc:377] Logged in from keytab as oryx/127.20.115.252@KRBTEST.COM (short username oryx)
I20260924 21:17:13.616921 27688 webserver.cc:460] Webserver started at http://127.20.115.252:37827/ using document root <none> and password file <none>
I20260924 21:17:13.617151 27688 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.617216 27688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.617350 27688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:13.618403 27688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data/instance:
uuid: "1ad70823ebb04903b0c5b677684c117b"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "d4925bf930f3e743a8e8e65f8a9164da"
server_key_iv: "721d619a53561c14817266124f9604a1"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.618762 27688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal/instance:
uuid: "1ad70823ebb04903b0c5b677684c117b"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "d4925bf930f3e743a8e8e65f8a9164da"
server_key_iv: "721d619a53561c14817266124f9604a1"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.620021 27688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:13.620723 27703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.620905 27688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:13.621001 27688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal
uuid: "1ad70823ebb04903b0c5b677684c117b"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "d4925bf930f3e743a8e8e65f8a9164da"
server_key_iv: "721d619a53561c14817266124f9604a1"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.621078 27688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.638958 27688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:13.640633 27688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:13.640832 27688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:13.645097 27688 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.252:32805
I20260924 21:17:13.645141 27755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.252:32805 every 8 connection(s)
I20260924 21:17:13.645495 27688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
I20260924 21:17:13.646607 27756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:13.648262 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 27688
I20260924 21:17:13.648623 27756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.252@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284632, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.115.252@KRBTEST.COM
I20260924 21:17:13.657567 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.649443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45350 (local address 127.20.115.254:40423)
0924 21:17:13.649503 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.649506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.649517 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:13.649775 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.649776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.649788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.649857 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.650414 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.650923 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.651794 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.652008 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.652094 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.654489 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.654491 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.654491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.654499 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.656145 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.656540 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.656541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.656542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.656602 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.656873 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.656874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.656874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.657004 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.657054 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.657380 (+ 326us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.657478 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:17:13.657686 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.649269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45350)
0924 21:17:13.649403 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.649438 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.649675 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.649878 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.649881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.650149 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.650308 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.650312 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.651021 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.651022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.651695 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.651696 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.652123 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.652125 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.652225 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.652724 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.652733 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.654395 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.656230 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.656235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.656237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.656464 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.656675 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.656675 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.656676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.656764 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.657070 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.657077 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.657231 (+ 154us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.657350 (+ 119us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.657503 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260924 21:17:13.658248 27756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:17:13.658437 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.648879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41302 (local address 127.20.115.252:32805)
0924 21:17:13.649100 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.649109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.649138 (+ 29us) server_negotiation.cc:407] Connection header received
0924 21:17:13.649659 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.649666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.649691 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.649797 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.650372 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.651155 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.651842 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.652169 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.652251 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.655002 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.655009 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.655010 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.655016 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.656753 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.657178 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.657180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.657180 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.657226 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.657429 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.657429 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.657430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.657588 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.658195 (+ 607us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.658238 (+ 43us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.658305 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":125,"threads_started":1}
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.252@KRBTEST.COM for oryx/127.20.115.253@KRBTEST.COM
I20260924 21:17:13.664629 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.658496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.0.0.1:42028)
0924 21:17:13.658665 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.658672 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.658732 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.658856 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.658938 (+ 82us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.659084 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.659118 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.659119 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.659760 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.659761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.660257 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.660259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.660655 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.660656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.660718 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.661134 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.661138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.662230 (+ 1092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.663586 (+ 1356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.663587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.663587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.663748 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.664049 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.664050 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.664050 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.664104 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.664405 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.664406 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.664441 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.664505 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.664572 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260924 21:17:13.664688 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.658555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42028 (local address 127.20.115.253:41823)
0924 21:17:13.658607 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.658609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.658722 (+ 113us) server_negotiation.cc:407] Connection header received
0924 21:17:13.658794 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.658795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.658806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.658856 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.659195 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.659679 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.660354 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.660569 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.660655 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.662306 (+ 1651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.662308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.662308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.662315 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.663489 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.663874 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.663875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.663875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.663974 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.664201 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.664202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.664203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.664321 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.664378 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.664564 (+ 186us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.664629 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:17:13.665344 27756 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.252@KRBTEST.COM for oryx/127.20.115.252@KRBTEST.COM
I20260924 21:17:13.670800 27759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.665599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:32805 (local address 127.0.0.1:41312)
0924 21:17:13.665643 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.665653 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.665696 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.665784 (+ 88us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.665785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.665886 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.665910 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.665911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.666610 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.666611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.667079 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.667080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.667420 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.667421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.667457 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.667766 (+ 309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.667769 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.668879 (+ 1110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.670130 (+ 1251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.670131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.670131 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.670278 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.670428 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.670429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.670429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.670470 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.670615 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.670615 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.670643 (+ 28us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.670686 (+ 43us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.670746 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 21:17:13.670858 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.665644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41312 (local address 127.20.115.252:32805)
0924 21:17:13.665701 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.665702 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.665711 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:13.665728 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.665728 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.665735 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.665776 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.665969 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.666514 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.667145 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.667371 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.667420 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.669018 (+ 1598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.669019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.669019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.669024 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.670070 (+ 1046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.670343 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.670344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.670344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.670388 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.670503 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.670503 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.670503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.670581 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.670623 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.670700 (+ 77us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.670787 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":36}
I20260924 21:17:13.672199 27756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: Bootstrap starting.
I20260924 21:17:13.673080 27756 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:13.673347 27756 log.cc:824] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:13.674091 27756 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: No bootstrap required, opened a new log
I20260924 21:17:13.675531 27756 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } }
I20260924 21:17:13.675681 27756 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:13.675729 27756 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ad70823ebb04903b0c5b677684c117b, State: Initialized, Role: FOLLOWER
I20260924 21:17:13.675863 27756 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } }
I20260924 21:17:13.676157 27756 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:13.676149 27761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } } }
I20260924 21:17:13.676291 27761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:17:13.677973 27772 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:17:13.678033 27772 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:17:13.680469 27688 master_runner.cc:428] Detected that this master 1ad70823ebb04903b0c5b677684c117b is joining an existing cluster
I20260924 21:17:13.680521 27688 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.115.252:32805
I20260924 21:17:13.688627 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.682121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41324 (local address 127.20.115.252:32805)
0924 21:17:13.682154 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.682156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.682510 (+ 354us) server_negotiation.cc:407] Connection header received
0924 21:17:13.682591 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.682592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.682601 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.682627 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.683011 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.683662 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.684576 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.684866 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.684906 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.686079 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.686080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.686081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.686088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.687559 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.687869 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.687870 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.687870 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.687937 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.688130 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.688131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.688131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.688262 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.688332 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.688432 (+ 100us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.688543 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":16}
I20260924 21:17:13.688764 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.681919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:32805 (local address 127.0.0.1:41324)
0924 21:17:13.682451 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.682461 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.682530 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.682674 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.682674 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.682872 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.682922 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.682924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.683730 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.683731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.684501 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.684502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.685074 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.685075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.685148 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.685596 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.685600 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.685928 (+ 328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.687642 (+ 1714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.687643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.687643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.687799 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.688017 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.688018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.688018 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.688067 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.688327 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.688327 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.688359 (+ 32us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.688408 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.688484 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":107,"threads_started":1}
I20260924 21:17:13.709791 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 27097
I20260924 21:17:13.714620 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.682302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45362 (local address 127.20.115.254:40423)
0924 21:17:13.682348 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.682350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.682519 (+ 169us) server_negotiation.cc:407] Connection header received
0924 21:17:13.682612 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.682613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.682623 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.682652 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.683081 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.683823 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.684872 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.685170 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.685212 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.711526 (+ 26314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.711528 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.711528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.711536 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.713346 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.713744 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.713745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.713745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.713800 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.714012 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.714013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.714014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.714159 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.714199 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.714395 (+ 196us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.714480 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:17:13.714788 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.682132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42044 (local address 127.20.115.253:41823)
0924 21:17:13.682169 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.682172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.682179 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:17:13.682200 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.682201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.682211 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.682263 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.682721 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.683500 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.684581 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.684997 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.685037 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.711500 (+ 26463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.711502 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.711502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.711511 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.713626 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.714026 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.714027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.714028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.714079 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.714265 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.714266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.714266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.714389 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.714428 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.714621 (+ 193us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.714697 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:17:13.715068 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.682268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45362)
0924 21:17:13.682457 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.682467 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.682542 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.682717 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.682717 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.682919 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.682990 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.682992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.683919 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.683919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.684745 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.684747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.685294 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.685295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.685353 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.709393 (+ 24040us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.709399 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.709816 (+ 417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.713434 (+ 3618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.713435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.713436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.713664 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.713866 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.713867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.713867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.713935 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.714242 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.714243 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.714288 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.714739 (+ 451us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.714882 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":122,"threads_started":1}
I20260924 21:17:13.715142 27778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.681916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.0.0.1:42044)
0924 21:17:13.682076 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.682088 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.682142 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.682311 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.682312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.682553 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.682631 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.682633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.683582 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.683583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.684488 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.684490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.685214 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.685215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.685278 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.710992 (+ 25714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.710997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.711398 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.713722 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.713723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.713724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.713942 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.714147 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.714148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.714148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.714199 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.714476 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.714476 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.714518 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.714883 (+ 365us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.714995 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
I20260924 21:17:13.717308 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:42089
--local_ip_for_outbound_sockets=127.20.115.193
--tserver_master_addrs=127.20.115.254:40423
--webserver_port=35211
--webserver_interface=127.20.115.193
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab
--principal=oryx/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.115.254:40423,127.20.115.253:41823
--tserver_master_addrs=127.20.115.254:40423,127.20.115.253:41823,127.20.115.252:32805 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284632, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.254@KRBTEST.COM for oryx/127.20.115.252@KRBTEST.COM
I20260924 21:17:13.726002 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.718331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:32805 (local address 127.0.0.1:41340)
0924 21:17:13.718376 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.718386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.718419 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.718583 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.718584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.718747 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.718790 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.718792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.719694 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.719695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.720533 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.720535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.721053 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.721054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.721117 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.721512 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.721517 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.723132 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.724976 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.724977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.724978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.725179 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.725383 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.725384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.725384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.725438 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.725691 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.725691 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.725732 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.725791 (+ 59us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.725901 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 21:17:13.726354 27758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.718385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41340 (local address 127.20.115.252:32805)
0924 21:17:13.718422 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.718423 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.718432 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:13.718498 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.718499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.718511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.718569 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.718875 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.719604 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.720622 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.720945 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.721004 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.723229 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.723230 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.723230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.723237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.724885 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.725253 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.725254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.725254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.725302 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.725500 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.725500 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.725501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.725613 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.725667 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.726172 (+ 505us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.726241 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:17:13.726933 27035 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.115.252:32805: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "1ad70823ebb04903b0c5b677684c117b"
member_type: NON_VOTER
last_known_addr {
host: "127.20.115.252"
port: 32805
}
attrs {
promote: true
}
}
dest_uuid: "da840c06cc98432690b005cdba3e5e4d"
cas_config_opid_index: 5
I20260924 21:17:13.727171 27035 consensus_queue.cc:237] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } }
I20260924 21:17:13.727517 27595 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1ad70823ebb04903b0c5b677684c117b (127.20.115.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.728439 27595 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.115.252:32805
I20260924 21:17:13.728507 27595 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.728569 27595 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:13.728921 27730 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:13.729787 27730 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Refusing update from remote peer da840c06cc98432690b005cdba3e5e4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 21:17:13.730263 27596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:13.730520 27688 master.cc:537] Master@127.20.115.252:32805 shutting down...
I20260924 21:17:13.731362 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.727556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.0.0.1:42052)
0924 21:17:13.727770 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.727779 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.727814 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.728011 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.728012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.728014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:13.728055 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.728056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.728995 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.728996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.730175 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.730177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.731114 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.731115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.731176 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.731198 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.731249 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.731282 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260924 21:17:13.731598 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.727641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.20.115.253:41823)
0924 21:17:13.727747 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.727749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.727864 (+ 115us) server_negotiation.cc:407] Connection header received
0924 21:17:13.727885 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.727886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.727908 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.727962 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:13.728130 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.728901 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.730274 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.730877 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.730948 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.731021 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.731496 (+ 475us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.731537 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260924 21:17:13.731828 27644 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 LEARNER]: Refusing update from remote peer da840c06cc98432690b005cdba3e5e4d: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260924 21:17:13.732118 27596 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260924 21:17:13.732664 27644 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 1ad70823ebb04903b0c5b677684c117b (127.20.115.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
W20260924 21:17:13.732952 27020 consensus_peers.cc:487] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d -> Peer 1ad70823ebb04903b0c5b677684c117b (127.20.115.252:32805): Couldn't send request to peer 1ad70823ebb04903b0c5b677684c117b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 21:17:13.734030 27674 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: SysCatalogTable state changed. Reason: New leader da840c06cc98432690b005cdba3e5e4d. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.734113 27674 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: This master's current role is: LEARNER
I20260924 21:17:13.734148 27674 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: NON_VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: true } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.734190 27674 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: This master's current role is: LEARNER
I20260924 21:17:13.734440 27596 raft_consensus.cc:1050] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: attempting to promote NON_VOTER b85dbe21632b4af8951f35fa0dac492d to VOTER
I20260924 21:17:13.734619 27596 consensus_queue.cc:237] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } }
I20260924 21:17:13.735468 27644 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 LEARNER]: Refusing update from remote peer da840c06cc98432690b005cdba3e5e4d: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260924 21:17:13.735749 27595 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260924 21:17:13.736748 27020 consensus_peers.cc:487] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d -> Peer 1ad70823ebb04903b0c5b677684c117b (127.20.115.252:32805): Couldn't send request to peer 1ad70823ebb04903b0c5b677684c117b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260924 21:17:13.737116 27595 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b85dbe21632b4af8951f35fa0dac492d (127.20.115.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.737351 27688 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:13.737428 27688 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:13.737468 27688 tablet_replica.cc:330] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: stopping tablet replica
I20260924 21:17:13.737931 27595 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.737995 27595 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:13.738044 27595 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.738082 27595 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:13.738564 27644 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b85dbe21632b4af8951f35fa0dac492d (127.20.115.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.739351 27783 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.739435 27783 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:17:13.750180 27688 master.cc:552] Master@127.20.115.252:32805 shutdown complete.
I20260924 21:17:13.750231 27688 master_runner.cc:305] Clearing existing system tablet
I20260924 21:17:13.750353 27688 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.750399 27688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.751142 27688 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:13.751706 27785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.751817 27688 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.751865 27688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.751886 27688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal
uuid: "1ad70823ebb04903b0c5b677684c117b"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "d4925bf930f3e743a8e8e65f8a9164da"
server_key_iv: "721d619a53561c14817266124f9604a1"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.752589 27688 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:13.754072 27688 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260924 21:17:13.754145 27688 log.cc:1196] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260924 21:17:13.754361 27688 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: Deleting consensus metadata
I20260924 21:17:13.754491 27688 master_runner.cc:315] Copying system tablet from 127.20.115.254:40423
I20260924 21:17:13.755100 27688 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: tablet copy: Beginning tablet copy session from remote peer at address 127.20.115.254:40423
I20260924 21:17:13.762248 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.755456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45364 (local address 127.20.115.254:40423)
0924 21:17:13.755503 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.755505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.755555 (+ 50us) server_negotiation.cc:407] Connection header received
0924 21:17:13.755618 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.755619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.755629 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.755711 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.756061 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.756881 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.757886 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.758183 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.758259 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.759347 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.759349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.759349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.759357 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.761039 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.761420 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.761421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.761422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.761487 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.761701 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.761702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.761702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.761835 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.761910 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.762081 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.762161 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:17:13.762281 27791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.755385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45364)
0924 21:17:13.755499 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.755509 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.755565 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.755717 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.755718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.755911 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.755961 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.755963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.756967 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.756967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.757789 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.757791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.758269 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.758270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.758326 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.758840 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.758845 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.759244 (+ 399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.761123 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.761124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.761125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.761337 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.761563 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.761564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.761564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.761629 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.761913 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.761914 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.761959 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.762036 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.762141 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
I20260924 21:17:13.762815 27055 tablet_copy_service.cc:140] P da840c06cc98432690b005cdba3e5e4d: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 1ad70823ebb04903b0c5b677684c117b ({username='oryx', principal='oryx/127.20.115.252@KRBTEST.COM'} at 127.0.0.1:45364)
I20260924 21:17:13.762892 27055 tablet_copy_service.cc:161] P da840c06cc98432690b005cdba3e5e4d: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 1ad70823ebb04903b0c5b677684c117b at {username='oryx', principal='oryx/127.20.115.252@KRBTEST.COM'} at 127.0.0.1:45364: session id = 1ad70823ebb04903b0c5b677684c117b-00000000000000000000000000000000
I20260924 21:17:13.763650 27055 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: Tablet Copy: opened 0 blocks and 1 log segments
I20260924 21:17:13.764074 27688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260924 21:17:13.765424 27688 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: tablet copy: Starting download of 0 data blocks...
I20260924 21:17:13.765743 27688 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: tablet copy: Starting download of 1 WAL segments...
I20260924 21:17:13.766635 27688 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260924 21:17:13.767580 27055 tablet_copy_service.cc:342] P da840c06cc98432690b005cdba3e5e4d: Request end of tablet copy session 1ad70823ebb04903b0c5b677684c117b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.115.252@KRBTEST.COM'} at 127.0.0.1:45364
I20260924 21:17:13.767632 27055 tablet_copy_service.cc:433] P da840c06cc98432690b005cdba3e5e4d: ending tablet copy session 1ad70823ebb04903b0c5b677684c117b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 1ad70823ebb04903b0c5b677684c117b
I20260924 21:17:13.770124 27688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:13.771186 27795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.771361 27792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.771430 27688 server_base.cc:1034] running on GCE node
W20260924 21:17:13.773531 27793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.773670 27688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:13.773792 27688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:13.774920 27688 hybrid_clock.cc:648] HybridClock initialized: now 1790284633774909 us; error 29 us; skew 500 ppm
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:13.776412 27688 init.cc:377] Logged in from keytab as oryx/127.20.115.252@KRBTEST.COM (short username oryx)
I20260924 21:17:13.776667 27688 webserver.cc:460] Webserver started at http://127.20.115.252:46663/ using document root <none> and password file <none>
I20260924 21:17:13.776808 27688 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.776847 27688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.777576 27688 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:13.778183 27802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.778288 27688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:13.778321 27688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal
uuid: "1ad70823ebb04903b0c5b677684c117b"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "d4925bf930f3e743a8e8e65f8a9164da"
server_key_iv: "721d619a53561c14817266124f9604a1"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.778371 27688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.797202 27688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:13.799104 27688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:13.802628 27688 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.252:32805
I20260924 21:17:13.802946 27688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
I20260924 21:17:13.803968 27855 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:17:13.804523 27855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: Bootstrap starting.
I20260924 21:17:13.804903 27854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.252:32805 every 8 connection(s)
I20260924 21:17:13.807494 27855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:17:13.807765 27855 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: Bootstrap complete.
I20260924 21:17:13.807955 27855 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } }
I20260924 21:17:13.808049 27855 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 1ad70823ebb04903b0c5b677684c117b, State: Initialized, Role: LEARNER
I20260924 21:17:13.808094 27855 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } }
I20260924 21:17:13.808283 27855 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:13.808276 27858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:13.808549 27858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: This master's current role is: LEARNER
I20260924 21:17:13.809749 27869 catalog_manager.cc:1259] Loaded cluster ID: 8353c0d9a88f4474ba76971f10fd583e
I20260924 21:17:13.809834 27869 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: loading cluster ID for follower catalog manager: success
I20260924 21:17:13.811538 27869 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: acquiring CA information for follower catalog manager: success
I20260924 21:17:13.812018 27869 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260924 21:17:13.821130 27781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:13.821306 27781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:13.821328 27781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:13.822456 27781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:13.822500 27781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:13.822526 27781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:13.823951 27781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab
--principal=oryx/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:42089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=35211
--tserver_master_addrs=127.20.115.254:40423,127.20.115.253:41823,127.20.115.252:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:13.824182 27781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:13.824411 27781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:13.829421 27877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.829443 27875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.829447 27874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.829730 27781 server_base.cc:1034] running on GCE node
I20260924 21:17:13.829937 27781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:13.830173 27781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:13.831342 27781 hybrid_clock.cc:648] HybridClock initialized: now 1790284633831326 us; error 29 us; skew 500 ppm
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:13.834033 27781 init.cc:377] Logged in from keytab as oryx/127.20.115.193@KRBTEST.COM (short username oryx)
I20260924 21:17:13.834439 27781 webserver.cc:460] Webserver started at http://127.20.115.193:35211/ using document root <none> and password file <none>
I20260924 21:17:13.834673 27781 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.834757 27781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.836097 27781 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:13.836688 27884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.836910 27781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:13.837030 27781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "7d4cf1aa4744407db2cb3454411b7c00"
format_stamp: "Formatted at 2026-09-24 21:17:12 on dist-test-slave-cxpd"
server_key: "bfc03db763a5b94a154057c00964d75f"
server_key_iv: "ed9e0e20a81eae4c0e12274d165c8680"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.837365 27781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.849464 27781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:13.851682 27781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:13.851833 27781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:13.852062 27781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:13.852368 27781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:13.852429 27781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.852473 27781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:13.852501 27781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.858727 27781 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:42089
I20260924 21:17:13.858770 27997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:42089 every 8 connection(s)
I20260924 21:17:13.859097 27781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:13.866420 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 27781
I20260924 21:17:13.866577 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 27231
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.252@KRBTEST.COM
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.253@KRBTEST.COM
I20260924 21:17:13.870183 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.859758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:59755 (local address 127.20.115.254:40423)
0924 21:17:13.859805 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.859807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.860188 (+ 381us) server_negotiation.cc:407] Connection header received
0924 21:17:13.861269 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.861271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.861285 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.861347 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.862057 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.862822 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.863724 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.864021 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.864082 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.866592 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.866593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.866594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.866603 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.868524 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.869034 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.869035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.869036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.869097 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.869410 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.869411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.869412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.869563 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.869615 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.869993 (+ 378us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.870077 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:17:13.870638 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.859778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.193:59755)
0924 21:17:13.860040 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.860086 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.861142 (+ 1056us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.861429 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.861434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.861826 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.861948 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.861955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.862911 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.862911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.863630 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.863632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.864148 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.864149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.864285 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.864839 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.864848 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.866462 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.868614 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.868621 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.868623 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.868958 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.869167 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.869168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.869169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.869310 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.869649 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.869655 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.869817 (+ 162us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.869920 (+ 103us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.870098 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":5,"thread_start_us":87,"threads_started":1}
I20260924 21:17:13.870792 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.860106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:56469 (local address 127.20.115.252:32805)
0924 21:17:13.860263 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.860265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.860273 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:13.861325 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.861327 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.861344 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.861402 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.862055 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.863069 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.864054 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.864432 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.864512 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.868066 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.868067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.868068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.868076 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.869631 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.869997 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.869999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.869999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.870058 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.870244 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.870245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.870245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.870372 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.870463 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.870625 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.870703 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":81,"threads_started":1}
I20260924 21:17:13.870790 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.859781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:32805 (local address 127.20.115.193:56469)
0924 21:17:13.860088 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.860096 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.861142 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.861451 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.861452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.861829 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.861948 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.861955 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.863151 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.863152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.863966 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.863967 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.864570 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.864571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.864626 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.865194 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.865198 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.867978 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.869731 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.869732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.869732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.869896 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.870125 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.870126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.870126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.870179 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.870458 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.870459 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.870501 (+ 42us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.870586 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.870685 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":39,"thread_start_us":91,"threads_started":1}
I20260924 21:17:13.871574 27998 heartbeater.cc:347] Connected to a master server at 127.20.115.252:32805
I20260924 21:17:13.871672 27998 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:13.871861 27998 heartbeater.cc:511] Master 127.20.115.252:32805 requested a full tablet report, sending...
I20260924 21:17:13.872350 27819 ts_manager.cc:194] Registered new tserver with Master: 7d4cf1aa4744407db2cb3454411b7c00 (127.20.115.193:42089)
I20260924 21:17:13.873514 27999 heartbeater.cc:347] Connected to a master server at 127.20.115.254:40423
I20260924 21:17:13.873570 27999 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:13.873637 27999 heartbeater.cc:511] Master 127.20.115.254:40423 requested a full tablet report, sending...
I20260924 21:17:13.873919 27035 ts_manager.cc:194] Re-registered known tserver with Master: 7d4cf1aa4744407db2cb3454411b7c00 (127.20.115.193:42089)
I20260924 21:17:13.874042 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:35431
--local_ip_for_outbound_sockets=127.20.115.194
--tserver_master_addrs=127.20.115.254:40423
--webserver_port=45359
--webserver_interface=127.20.115.194
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab
--principal=oryx/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.115.254:40423,127.20.115.253:41823
--tserver_master_addrs=127.20.115.254:40423,127.20.115.253:41823,127.20.115.252:32805 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:17:13.874467 27035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:59755
I20260924 21:17:13.876561 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.859884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:52661 (local address 127.20.115.253:41823)
0924 21:17:13.859926 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.859928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.860324 (+ 396us) server_negotiation.cc:407] Connection header received
0924 21:17:13.861311 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.861313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.861325 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.861367 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.862053 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.862605 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.863802 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.864098 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.864163 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.873717 (+ 9554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.873718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.873719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.873728 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:13.875399 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.875758 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.875759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.875760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.875815 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:13.876025 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:13.876026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:13.876026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:13.876152 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:13.876228 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:13.876372 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 21:17:13.876466 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:17:13.876573 28004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.859782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.20.115.193:52661)
0924 21:17:13.860272 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.860280 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.861166 (+ 886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.861449 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.861450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.861757 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.861947 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.861952 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.862716 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.862717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.863731 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.863733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.864168 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.864169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.864286 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.865054 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.865060 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.873591 (+ 8531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:13.875501 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.875502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.875503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.875671 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.875890 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:13.875890 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:13.875891 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:13.875955 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:13.876221 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:13.876222 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:13.876262 (+ 40us) client_negotiation.cc:769] Sending connection context
0924 21:17:13.876349 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:17:13.876455 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":94,"thread_start_us":53,"threads_started":1}
I20260924 21:17:13.876896 28000 heartbeater.cc:347] Connected to a master server at 127.20.115.253:41823
I20260924 21:17:13.876976 28000 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:13.877046 28000 heartbeater.cc:511] Master 127.20.115.253:41823 requested a full tablet report, sending...
I20260924 21:17:13.877426 27627 ts_manager.cc:194] Registered new tserver with Master: 7d4cf1aa4744407db2cb3454411b7c00 (127.20.115.193:42089)
W20260924 21:17:13.954741 28006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:13.954996 28006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:13.955019 28006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:13.956086 28006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:13.956118 28006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:13.956143 28006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:13.957523 28006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab
--principal=oryx/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:35431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=45359
--tserver_master_addrs=127.20.115.254:40423,127.20.115.253:41823,127.20.115.252:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:13.957695 28006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:13.957860 28006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:13.962704 28011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:13.962711 28012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.962802 28006 server_base.cc:1034] running on GCE node
W20260924 21:17:13.962711 28014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:13.963151 28006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:13.963344 28006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:13.964488 28006 hybrid_clock.cc:648] HybridClock initialized: now 1790284633964475 us; error 36 us; skew 500 ppm
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:13.967022 28006 init.cc:377] Logged in from keytab as oryx/127.20.115.194@KRBTEST.COM (short username oryx)
I20260924 21:17:13.967454 28006 webserver.cc:460] Webserver started at http://127.20.115.194:45359/ using document root <none> and password file <none>
I20260924 21:17:13.967679 28006 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:13.967751 28006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:13.969237 28006 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:13.969810 28021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.970012 28006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:13.970099 28006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "0edc40f3b1034ae9806e1cb201c1c793"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "36c0e0603dc40cf21f07f1d0616f4d1c"
server_key_iv: "045e25700e8b9032f40caeb3b258c4c6"
server_key_version: "encryptionkey@0"
I20260924 21:17:13.970402 28006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:13.982087 28006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:13.984033 28006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:13.984184 28006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:13.984434 28006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:13.984726 28006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:13.984760 28006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.984849 28006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:13.984876 28006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:13.990832 28006 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:35431
I20260924 21:17:13.990872 28134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:35431 every 8 connection(s)
I20260924 21:17:13.991219 28006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.253@KRBTEST.COM
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
Sep 24 21:17:13 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284633, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.252@KRBTEST.COM
I20260924 21:17:13.999310 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 28006
I20260924 21:17:13.999480 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 27365
I20260924 21:17:14.002465 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.992166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:40041 (local address 127.20.115.253:41823)
0924 21:17:13.992216 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.992219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.992229 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:13.993381 (+ 1152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.993383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.993396 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.993442 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.994177 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.994799 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.995855 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.996099 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.996158 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.998475 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.998477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.998477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.998485 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.000762 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.001321 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.001322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.001323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.001384 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.001676 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.001677 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.001677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.001822 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.001866 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.002217 (+ 351us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.002305 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:17:14.002461 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.992165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:36079 (local address 127.20.115.252:32805)
0924 21:17:13.992257 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.992259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.992441 (+ 182us) server_negotiation.cc:407] Connection header received
0924 21:17:13.993402 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.993404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.993416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.993447 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.994199 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.995008 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.996124 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.996428 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.996505 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.999522 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.999524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.999524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.999533 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.001199 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.001600 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.001601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.001601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.001659 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.001861 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.001862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.001862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.001994 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.002041 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.002260 (+ 219us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.002351 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 21:17:14.002803 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.992276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:59527 (local address 127.20.115.254:40423)
0924 21:17:13.992319 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:13.992321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:13.992328 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:17:13.993376 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:13.993378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:13.993393 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:13.993438 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:13.994197 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.994985 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.996042 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.996298 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.996382 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.998946 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:13.998947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:13.998947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:13.998955 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.001578 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.001951 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.001952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.001952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.002000 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.002204 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.002204 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.002205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.002327 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.002367 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.002610 (+ 243us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.002692 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:17:14.003057 28139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.991920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.20.115.194:40041)
0924 21:17:13.992104 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.992145 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.993248 (+ 1103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.993527 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.993530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.993852 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.994087 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.994091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.994912 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.994913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.995763 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.995765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.996171 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.996171 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.996301 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.996811 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.996820 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.998387 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:14.000892 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.000899 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.000901 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.001228 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.001469 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.001470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.001471 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.001601 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.001910 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:14.001915 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:14.002071 (+ 156us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.002545 (+ 474us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.002772 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260924 21:17:14.003064 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.991921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.194:59527)
0924 21:17:13.992167 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.992178 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.993248 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.993526 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.993531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.993889 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.994087 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.994093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.995089 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.995090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.995958 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.995960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.996375 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.996376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.996458 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.996990 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.996996 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.998833 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:14.001658 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.001658 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.001659 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.001860 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.002073 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.002074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.002074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.002127 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.002415 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:14.002416 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:14.002451 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.002891 (+ 440us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.002976 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":1,"thread_start_us":109,"threads_started":1}
I20260924 21:17:14.003064 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:13.991920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:32805 (local address 127.20.115.194:36079)
0924 21:17:13.992384 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:13.992393 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:13.993248 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:13.993526 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:13.993531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:13.993858 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:13.994087 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.994093 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.995270 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.995270 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.996060 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:13.996061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:13.996560 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:13.996560 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:13.996604 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:13.997143 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:13.997148 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:13.999411 (+ 2263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:14.001286 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.001287 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.001287 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.001513 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.001734 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.001735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.001735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.001787 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.002088 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:14.002088 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:14.002127 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.002549 (+ 422us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.002851 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":154,"thread_start_us":69,"threads_started":1}
I20260924 21:17:14.003450 28137 heartbeater.cc:347] Connected to a master server at 127.20.115.253:41823
I20260924 21:17:14.003515 28137 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:14.003640 28137 heartbeater.cc:511] Master 127.20.115.253:41823 requested a full tablet report, sending...
I20260924 21:17:14.004067 27627 ts_manager.cc:194] Registered new tserver with Master: 0edc40f3b1034ae9806e1cb201c1c793 (127.20.115.194:35431)
I20260924 21:17:14.005308 28136 heartbeater.cc:347] Connected to a master server at 127.20.115.254:40423
I20260924 21:17:14.005359 28136 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:14.005435 28136 heartbeater.cc:511] Master 127.20.115.254:40423 requested a full tablet report, sending...
I20260924 21:17:14.005471 28135 heartbeater.cc:347] Connected to a master server at 127.20.115.252:32805
I20260924 21:17:14.005518 28135 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:14.005585 28135 heartbeater.cc:511] Master 127.20.115.252:32805 requested a full tablet report, sending...
I20260924 21:17:14.005683 27035 ts_manager.cc:194] Re-registered known tserver with Master: 0edc40f3b1034ae9806e1cb201c1c793 (127.20.115.194:35431)
I20260924 21:17:14.006163 27035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:59527
I20260924 21:17:14.006485 27819 ts_manager.cc:194] Registered new tserver with Master: 0edc40f3b1034ae9806e1cb201c1c793 (127.20.115.194:35431)
I20260924 21:17:14.006613 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:45431
--local_ip_for_outbound_sockets=127.20.115.195
--tserver_master_addrs=127.20.115.254:40423
--webserver_port=33761
--webserver_interface=127.20.115.195
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab
--principal=oryx/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.115.254:40423,127.20.115.253:41823
--tserver_master_addrs=127.20.115.254:40423,127.20.115.253:41823,127.20.115.252:32805 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:14.088912 28142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:14.089164 28142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:14.089205 28142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:14.090390 28142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:14.090467 28142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:14.090514 28142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:14.092096 28142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab
--principal=oryx/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:45431
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=33761
--tserver_master_addrs=127.20.115.254:40423,127.20.115.253:41823,127.20.115.252:32805
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:14.092372 28142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:14.092613 28142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:14.098060 28150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:14.098099 28148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:14.098104 28147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:14.098140 28142 server_base.cc:1034] running on GCE node
I20260924 21:17:14.098440 28142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:14.098766 28142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:14.099936 28142 hybrid_clock.cc:648] HybridClock initialized: now 1790284634099922 us; error 27 us; skew 500 ppm
Sep 24 21:17:14 dist-test-slave-cxpd krb5kdc[26990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284634, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:14.102499 28142 init.cc:377] Logged in from keytab as oryx/127.20.115.195@KRBTEST.COM (short username oryx)
I20260924 21:17:14.102952 28142 webserver.cc:460] Webserver started at http://127.20.115.195:33761/ using document root <none> and password file <none>
I20260924 21:17:14.103135 28142 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:14.103184 28142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:14.104568 28142 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:14.105291 28157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:14.105444 28142 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:17:14.105537 28142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "1a7da08f0aa6420597f34f1333393d80"
format_stamp: "Formatted at 2026-09-24 21:17:13 on dist-test-slave-cxpd"
server_key: "c84af5aac66b517808d2c9ef812d4b59"
server_key_iv: "ce231d35cbb03ffd35ba8468153fc5bf"
server_key_version: "encryptionkey@0"
I20260924 21:17:14.105862 28142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:14.113787 28142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:14.115573 28142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:14.115676 28142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:14.115844 28142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:14.116086 28142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:14.116114 28142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:14.116135 28142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:14.116147 28142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:14.122264 28142 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:45431
I20260924 21:17:14.122349 28270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:45431 every 8 connection(s)
I20260924 21:17:14.122692 28142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:17:14 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284634, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.253@KRBTEST.COM
Sep 24 21:17:14 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284634, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.252@KRBTEST.COM
Sep 24 21:17:14 dist-test-slave-cxpd krb5kdc[26990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284634, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:14.131788 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 28142
I20260924 21:17:14.134745 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.123810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:52501 (local address 127.20.115.254:40423)
0924 21:17:14.123859 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.123861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.123936 (+ 75us) server_negotiation.cc:407] Connection header received
0924 21:17:14.125067 (+ 1131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.125068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.125080 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.125143 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:14.125957 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.126462 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.128212 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.128465 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.128531 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.131519 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:14.131520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:14.131521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:14.131528 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.133352 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.133742 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.133743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.133743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.133799 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.134022 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.134023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.134023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.134164 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.134204 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.134574 (+ 370us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.134656 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 21:17:14.134750 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.123587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:43303 (local address 127.20.115.253:41823)
0924 21:17:14.123694 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.123697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.123973 (+ 276us) server_negotiation.cc:407] Connection header received
0924 21:17:14.125199 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.125200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.125209 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.125270 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:14.125945 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.126751 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.127832 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.128118 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.128166 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.130489 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:14.130490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:14.130490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:14.130497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.133539 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.133920 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.133922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.133922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.133976 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.134176 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.134177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.134177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.134325 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.134393 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.134565 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.134658 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260924 21:17:14.134759 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.123640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:48141 (local address 127.20.115.252:32805)
0924 21:17:14.123733 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.123735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.124217 (+ 482us) server_negotiation.cc:407] Connection header received
0924 21:17:14.124999 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.125000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.125014 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.125073 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:14.125977 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.126738 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.127672 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.127925 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.127990 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.130965 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:14.130967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:14.130967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:14.130975 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.132843 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.133562 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.133563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.133564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.133628 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.133981 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.133982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.133983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.134112 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.134180 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.134597 (+ 417us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.134677 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260924 21:17:14.134971 28276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.123479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.20.115.195:43303)
0924 21:17:14.123822 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:14.123897 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:14.124893 (+ 996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:14.125276 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:14.125277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:14.125646 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:14.125842 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.125847 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.126819 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.126820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.127751 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.127753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.128185 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.128186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.128247 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.128912 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:14.128918 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:14.130392 (+ 1474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:14.133620 (+ 3228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.133621 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.133621 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.133838 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.134051 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.134051 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.134052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.134099 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.134405 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:14.134405 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:14.134439 (+ 34us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.134530 (+ 91us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.134698 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260924 21:17:14.135421 28277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.123479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:32805 (local address 127.20.115.195:48141)
0924 21:17:14.124149 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:14.124158 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:14.124875 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:14.125177 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:14.125181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:14.125687 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:14.125843 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.125849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.126853 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.126854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.127579 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.127581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.128004 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.128005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.128101 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.128720 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:14.128732 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:14.130871 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:14.132942 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.132950 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.132953 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.133465 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.133724 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.133725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.133725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.133888 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.134232 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:14.134238 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:14.134399 (+ 161us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.135200 (+ 801us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.135288 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":66,"spinlock_wait_cycles":3456,"thread_start_us":105,"threads_started":1}
I20260924 21:17:14.135589 28275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.123479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:52501)
0924 21:17:14.123813 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:14.123859 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:14.124869 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:14.125177 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:14.125181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:14.125686 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:14.125843 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.125849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.126614 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.126615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.127760 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.127762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.128552 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.128553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.128657 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.129071 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:14.129076 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:14.131451 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:14.133449 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.133450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.133450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.133659 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.133889 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:14.133889 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:14.133890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:14.133947 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:14.134252 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:14.134253 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:14.134399 (+ 146us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.135336 (+ 937us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.135417 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":49,"thread_start_us":101,"threads_started":1}
I20260924 21:17:14.136196 28271 heartbeater.cc:347] Connected to a master server at 127.20.115.252:32805
I20260924 21:17:14.136307 28271 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:14.136510 28271 heartbeater.cc:511] Master 127.20.115.252:32805 requested a full tablet report, sending...
I20260924 21:17:14.137003 28272 heartbeater.cc:347] Connected to a master server at 127.20.115.254:40423
I20260924 21:17:14.137048 28272 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:14.137105 28272 heartbeater.cc:511] Master 127.20.115.254:40423 requested a full tablet report, sending...
I20260924 21:17:14.137466 27819 ts_manager.cc:194] Registered new tserver with Master: 1a7da08f0aa6420597f34f1333393d80 (127.20.115.195:45431)
I20260924 21:17:14.137696 27035 ts_manager.cc:194] Re-registered known tserver with Master: 1a7da08f0aa6420597f34f1333393d80 (127.20.115.195:45431)
I20260924 21:17:14.138180 27035 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:52501
I20260924 21:17:14.139147 28273 heartbeater.cc:347] Connected to a master server at 127.20.115.253:41823
I20260924 21:17:14.139194 28273 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:14.139269 28273 heartbeater.cc:511] Master 127.20.115.253:41823 requested a full tablet report, sending...
I20260924 21:17:14.139619 27627 ts_manager.cc:194] Registered new tserver with Master: 1a7da08f0aa6420597f34f1333393d80 (127.20.115.195:45431)
I20260924 21:17:14.143644 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.133421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42058 (local address 127.20.115.253:41823)
0924 21:17:14.134888 (+ 1467us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.134890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.136033 (+ 1143us) server_negotiation.cc:407] Connection header received
0924 21:17:14.136064 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.136065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.136075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.136133 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:14.136965 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.137637 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.138636 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.138904 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.138945 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.140727 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:14.140728 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:14.140728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:14.140736 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.142461 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.142845 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.142846 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.142846 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.142905 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.143108 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.143109 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.143110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.143225 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.143327 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.143449 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.143519 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1452,"thread_start_us":48,"threads_started":1}
I20260924 21:17:14.144258 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.133646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41348 (local address 127.20.115.252:32805)
0924 21:17:14.134875 (+ 1229us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.134876 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.136158 (+ 1282us) server_negotiation.cc:407] Connection header received
0924 21:17:14.136250 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.136251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.136262 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.136293 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:14.137054 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.137752 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.138860 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.139145 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.139185 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.141408 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:14.141409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:14.141409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:14.141415 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.142941 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.143333 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.143335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.143335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.143390 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.143629 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.143630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.143630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.143777 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.143868 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.144068 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.144151 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1214,"thread_start_us":58,"threads_started":1}
I20260924 21:17:14.144596 27074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.133034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45376 (local address 127.20.115.254:40423)
0924 21:17:14.134846 (+ 1812us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.134848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.135614 (+ 766us) server_negotiation.cc:407] Connection header received
0924 21:17:14.135856 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.135857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.135870 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.135921 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:14.136920 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.137708 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.138852 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.139146 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.139185 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.141083 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:14.141085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:14.141085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:14.141093 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.143380 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.143791 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.143792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.143793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.143844 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.144060 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.144061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.144061 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.144197 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.144294 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.144404 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.144480 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1775,"thread_start_us":80,"threads_started":1}
I20260924 21:17:14.146099 27035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:17:14.147637 27035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:14.155493 27593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.151441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:35431 (local address 127.0.0.1:57004)
0924 21:17:14.151482 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:14.151494 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:14.151531 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:14.152135 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:14.152136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:14.152138 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:14.152184 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.152186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.153519 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.153520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.154425 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.154427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.155241 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.155242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.155300 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.155304 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.155370 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.155410 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":33}
I20260924 21:17:14.155517 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.151704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.20.115.194:35431)
0924 21:17:14.151915 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.151921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.151932 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:14.151980 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.151982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.152004 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.152140 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:14.152243 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.153412 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.154515 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.155153 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.155246 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.155302 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.155385 (+ 83us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.155421 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":93,"threads_started":1}
I20260924 21:17:14.155879 28292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.151538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:45431 (local address 127.0.0.1:41078)
0924 21:17:14.151662 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:14.151672 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:14.151890 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:14.152515 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:14.152516 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:14.152518 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:14.152556 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.152557 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.153442 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.153443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.154748 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.154750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.155632 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.155633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.155711 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.155717 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.155774 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.155816 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":40,"thread_start_us":95,"threads_started":1}
I20260924 21:17:14.155953 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.151954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.20.115.195:45431)
0924 21:17:14.152352 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.152357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.152367 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:14.152404 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.152406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.152420 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.152519 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:14.152609 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.153336 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.154852 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.155450 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.155541 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.155595 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.155809 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.155859 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":88,"threads_started":1}
I20260924 21:17:14.156061 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.151543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:42089 (local address 127.0.0.1:35942)
0924 21:17:14.151991 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:14.152001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:14.152053 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:14.152264 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:14.152265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:14.152268 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:14.152315 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.152317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.153521 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.153522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.154739 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.154741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.155709 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.155710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.155780 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.155785 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.155837 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.155876 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":16,"thread_start_us":82,"threads_started":1}
I20260924 21:17:14.156059 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.151749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35942 (local address 127.20.115.193:42089)
0924 21:17:14.151916 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.151921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.152062 (+ 141us) server_negotiation.cc:407] Connection header received
0924 21:17:14.152110 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.152112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.152128 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.152199 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:14.152392 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.153438 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.154839 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.155537 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.155625 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.155679 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.155916 (+ 237us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.155969 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":77,"threads_started":1}
I20260924 21:17:14.156558 28069 tablet_service.cc:1467] Processing CreateTablet for tablet 32f9d6979db04f40a5ff1cb262866d35 (DEFAULT_TABLE table=test-table [id=853d4d5179864e3c9e1f0305e03b6816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:14.156769 27932 tablet_service.cc:1467] Processing CreateTablet for tablet 32f9d6979db04f40a5ff1cb262866d35 (DEFAULT_TABLE table=test-table [id=853d4d5179864e3c9e1f0305e03b6816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:14.156955 28069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f9d6979db04f40a5ff1cb262866d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:14.157172 27932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f9d6979db04f40a5ff1cb262866d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:14.157380 28205 tablet_service.cc:1467] Processing CreateTablet for tablet 32f9d6979db04f40a5ff1cb262866d35 (DEFAULT_TABLE table=test-table [id=853d4d5179864e3c9e1f0305e03b6816]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:14.157693 28205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f9d6979db04f40a5ff1cb262866d35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:14.160153 28297 tablet_bootstrap.cc:492] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: Bootstrap starting.
I20260924 21:17:14.160490 28298 tablet_bootstrap.cc:492] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: Bootstrap starting.
I20260924 21:17:14.160921 28297 tablet_bootstrap.cc:654] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:14.161154 28298 tablet_bootstrap.cc:654] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:14.161281 28297 log.cc:824] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:14.161481 28298 log.cc:824] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:14.161800 28299 tablet_bootstrap.cc:492] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: Bootstrap starting.
I20260924 21:17:14.162124 28297 tablet_bootstrap.cc:492] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: No bootstrap required, opened a new log
I20260924 21:17:14.162178 28298 tablet_bootstrap.cc:492] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: No bootstrap required, opened a new log
I20260924 21:17:14.162196 28297 ts_tablet_manager.cc:1397] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:17:14.162225 28298 ts_tablet_manager.cc:1397] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:17:14.162393 28299 tablet_bootstrap.cc:654] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:14.162667 28299 log.cc:824] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:14.163335 28299 tablet_bootstrap.cc:492] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: No bootstrap required, opened a new log
I20260924 21:17:14.163417 28299 ts_tablet_manager.cc:1397] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:14.164227 28297 raft_consensus.cc:348] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:14.164397 28297 raft_consensus.cc:374] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:14.164445 28297 raft_consensus.cc:729] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4cf1aa4744407db2cb3454411b7c00, State: Initialized, Role: FOLLOWER
I20260924 21:17:14.164592 28297 consensus_queue.cc:260] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:14.164676 28298 raft_consensus.cc:348] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:14.164853 28298 raft_consensus.cc:374] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:14.164892 28297 ts_tablet_manager.cc:1428] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 21:17:14.164894 28298 raft_consensus.cc:729] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0edc40f3b1034ae9806e1cb201c1c793, State: Initialized, Role: FOLLOWER
I20260924 21:17:14.165040 27999 heartbeater.cc:503] Master 127.20.115.254:40423 was elected leader, sending a full tablet report...
I20260924 21:17:14.165017 28298 consensus_queue.cc:260] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:14.165256 28298 ts_tablet_manager.cc:1428] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260924 21:17:14.165755 28136 heartbeater.cc:503] Master 127.20.115.254:40423 was elected leader, sending a full tablet report...
I20260924 21:17:14.166946 28299 raft_consensus.cc:348] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:14.167094 28299 raft_consensus.cc:374] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:14.167137 28299 raft_consensus.cc:729] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a7da08f0aa6420597f34f1333393d80, State: Initialized, Role: FOLLOWER
I20260924 21:17:14.167244 28299 consensus_queue.cc:260] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:14.167477 28299 ts_tablet_manager.cc:1428] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260924 21:17:14.167542 28272 heartbeater.cc:503] Master 127.20.115.254:40423 was elected leader, sending a full tablet report...
W20260924 21:17:14.242646 28138 tablet.cc:2369] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:17:14.359725 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.357198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:32805 (local address 127.0.0.1:41364)
0924 21:17:14.357267 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:14.357280 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:14.357338 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:14.357516 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:14.357517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:14.357519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:14.357567 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.357569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.358175 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.358175 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.358887 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.358889 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.359500 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.359501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.359573 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.359577 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.359625 (+ 48us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.359663 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 21:17:14.359767 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.357302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.20.115.252:32805)
0924 21:17:14.357371 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.357373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.357384 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:14.357421 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.357422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.357441 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.357502 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:14.357624 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.358112 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.358971 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.359416 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.359475 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.359529 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.359664 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.359702 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260924 21:17:14.360064 28001 tablet.cc:2369] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:17:14.360235 27829 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260924 21:17:14.361191 28306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: SysCatalogTable state changed. Reason: New leader da840c06cc98432690b005cdba3e5e4d. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:14.361452 28306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: This master's current role is: LEARNER
W20260924 21:17:14.374321 28274 tablet.cc:2369] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:17:14.429692 28304 raft_consensus.cc:482] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:14.429814 28304 raft_consensus.cc:504] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:14.430240 28304 leader_election.cc:288] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a7da08f0aa6420597f34f1333393d80 (127.20.115.195:45431), 7d4cf1aa4744407db2cb3454411b7c00 (127.20.115.193:42089)
I20260924 21:17:14.433462 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.430564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:46157 (local address 127.20.115.195:45431)
0924 21:17:14.430612 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.430615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.430626 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:14.430689 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.430691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.430699 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.430776 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:14.430945 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.431510 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.432570 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.433069 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.433129 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.433137 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.433340 (+ 203us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.433393 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:17:14.433458 28141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.430442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:45431 (local address 127.20.115.194:46157)
0924 21:17:14.430563 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:14.430576 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:14.430627 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:14.430780 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:14.430781 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:14.430782 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:14.430873 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.430875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.431611 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.431612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.432474 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.432477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.433167 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.433168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.433250 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.433254 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.433320 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.433368 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":26}
I20260924 21:17:14.433665 28139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.430483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:42089 (local address 127.20.115.194:33319)
0924 21:17:14.430567 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:14.430577 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:14.430626 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:14.430788 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:14.430789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:14.430790 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:14.430881 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.430882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.431601 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.431602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.432555 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:14.432558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.433416 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.433417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:14.433491 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.433494 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:14.433549 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 21:17:14.433596 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68,"mutex_wait_us":86}
I20260924 21:17:14.433882 28295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.430557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:33319 (local address 127.20.115.193:42089)
0924 21:17:14.430603 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.430605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.430616 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:14.430689 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.430690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.430698 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.430774 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:14.430978 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.431485 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.432668 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.433337 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.433408 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.433415 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.433767 (+ 352us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.433817 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:17:14.433849 28225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f9d6979db04f40a5ff1cb262866d35" candidate_uuid: "0edc40f3b1034ae9806e1cb201c1c793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a7da08f0aa6420597f34f1333393d80" is_pre_election: true
I20260924 21:17:14.434022 28225 raft_consensus.cc:2454] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0edc40f3b1034ae9806e1cb201c1c793 in term 0.
I20260924 21:17:14.434157 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f9d6979db04f40a5ff1cb262866d35" candidate_uuid: "0edc40f3b1034ae9806e1cb201c1c793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d4cf1aa4744407db2cb3454411b7c00" is_pre_election: true
I20260924 21:17:14.434238 28024 leader_election.cc:302] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0edc40f3b1034ae9806e1cb201c1c793, 1a7da08f0aa6420597f34f1333393d80; no voters:
I20260924 21:17:14.434315 27952 raft_consensus.cc:2454] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0edc40f3b1034ae9806e1cb201c1c793 in term 0.
I20260924 21:17:14.434356 28304 raft_consensus.cc:2781] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:14.434404 28304 raft_consensus.cc:482] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:14.434432 28304 raft_consensus.cc:3037] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:14.435114 28304 raft_consensus.cc:504] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:14.435263 28304 leader_election.cc:288] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [CANDIDATE]: Term 1 election: Requested vote from peers 1a7da08f0aa6420597f34f1333393d80 (127.20.115.195:45431), 7d4cf1aa4744407db2cb3454411b7c00 (127.20.115.193:42089)
I20260924 21:17:14.435416 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f9d6979db04f40a5ff1cb262866d35" candidate_uuid: "0edc40f3b1034ae9806e1cb201c1c793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d4cf1aa4744407db2cb3454411b7c00"
I20260924 21:17:14.435482 28225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f9d6979db04f40a5ff1cb262866d35" candidate_uuid: "0edc40f3b1034ae9806e1cb201c1c793" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a7da08f0aa6420597f34f1333393d80"
I20260924 21:17:14.435518 27952 raft_consensus.cc:3037] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:14.435595 28225 raft_consensus.cc:3037] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:14.436225 27952 raft_consensus.cc:2454] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0edc40f3b1034ae9806e1cb201c1c793 in term 1.
I20260924 21:17:14.436259 28225 raft_consensus.cc:2454] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0edc40f3b1034ae9806e1cb201c1c793 in term 1.
I20260924 21:17:14.436419 28024 leader_election.cc:302] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0edc40f3b1034ae9806e1cb201c1c793, 1a7da08f0aa6420597f34f1333393d80; no voters:
I20260924 21:17:14.436547 28304 raft_consensus.cc:2781] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:14.436691 28304 raft_consensus.cc:686] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 1 LEADER]: Becoming Leader. State: Replica: 0edc40f3b1034ae9806e1cb201c1c793, State: Running, Role: LEADER
I20260924 21:17:14.436812 28304 consensus_queue.cc:237] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:14.437683 27035 catalog_manager.cc:5495] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0edc40f3b1034ae9806e1cb201c1c793 (127.20.115.194). New cstate: current_term: 1 leader_uuid: "0edc40f3b1034ae9806e1cb201c1c793" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } health_report { overall_health: UNKNOWN } } }
I20260924 21:17:14.438234 27594 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: NON_VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: true } } }
I20260924 21:17:14.438326 27594 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:14.439105 27596 raft_consensus.cc:1050] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: attempting to promote NON_VOTER 1ad70823ebb04903b0c5b677684c117b to VOTER
I20260924 21:17:14.439218 27596 consensus_queue.cc:237] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } }
I20260924 21:17:14.439589 27829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 LEARNER]: Refusing update from remote peer da840c06cc98432690b005cdba3e5e4d: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260924 21:17:14.439605 27644 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Refusing update from remote peer da840c06cc98432690b005cdba3e5e4d: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260924 21:17:14.439785 27594 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 21:17:14.439899 28313 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260924 21:17:14.440524 27594 raft_consensus.cc:2932] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1ad70823ebb04903b0c5b677684c117b (127.20.115.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:14.440685 27829 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1ad70823ebb04903b0c5b677684c117b (127.20.115.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:14.441463 27594 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:14.441555 27594 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:14.441555 28306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:14.441635 28306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:17:14.441614 27594 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:14.441663 27594 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:14.441670 27596 sys_catalog.cc:455] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:14.441798 27596 sys_catalog.cc:458] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:14.441762 27644 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 1ad70823ebb04903b0c5b677684c117b (127.20.115.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:14.442561 28315 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da840c06cc98432690b005cdba3e5e4d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:14.442669 28315 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:17:14.510469 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.507711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57020 (local address 127.20.115.194:35431)
0924 21:17:14.507779 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.507781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.507808 (+ 27us) server_negotiation.cc:407] Connection header received
0924 21:17:14.507857 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.507858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.507876 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.507951 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:14.508108 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.508761 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.509738 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.509949 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.510006 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.510147 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:14.510243 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:14.510304 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.510334 (+ 30us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.510350 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:17:14.512228 27952 raft_consensus.cc:1261] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [term 1 FOLLOWER]: Refusing update from remote peer 0edc40f3b1034ae9806e1cb201c1c793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:14.512228 28225 raft_consensus.cc:1261] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [term 1 FOLLOWER]: Refusing update from remote peer 0edc40f3b1034ae9806e1cb201c1c793: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:14.512611 28304 consensus_queue.cc:1035] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:14.512750 28311 consensus_queue.cc:1035] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:14.515646 28317 mvcc.cc:205] Tried to move back new op lower bound from 7333005862959202304 to 7333005862653734912. Current Snapshot: MvccSnapshot[applied={T|T < 7333005862959202304}]
I20260924 21:17:14.518671 27035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:14.518780 27035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:14.520273 27035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da840c06cc98432690b005cdba3e5e4d: Sending DeleteTablet for 3 replicas of tablet 32f9d6979db04f40a5ff1cb262866d35
I20260924 21:17:14.520713 27932 tablet_service.cc:1514] Processing DeleteTablet for tablet 32f9d6979db04f40a5ff1cb262866d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:14 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:35942
I20260924 21:17:14.521077 28205 tablet_service.cc:1514] Processing DeleteTablet for tablet 32f9d6979db04f40a5ff1cb262866d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:14 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:41078
I20260924 21:17:14.521323 28327 tablet_replica.cc:330] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: stopping tablet replica
I20260924 21:17:14.521307 28069 tablet_service.cc:1514] Processing DeleteTablet for tablet 32f9d6979db04f40a5ff1cb262866d35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:14 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:57004
I20260924 21:17:14.521427 28327 raft_consensus.cc:2229] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:14.521526 28327 raft_consensus.cc:2258] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:14.521530 28330 tablet_replica.cc:330] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: stopping tablet replica
I20260924 21:17:14.521627 28330 raft_consensus.cc:2229] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:17:14.521714 28330 raft_consensus.cc:2258] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:14.521842 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 27003
I20260924 21:17:14.522131 28330 ts_tablet_manager.cc:1905] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:14.524192 28330 ts_tablet_manager.cc:1918] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:14.524271 28330 log.cc:1196] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/32f9d6979db04f40a5ff1cb262866d35
I20260924 21:17:14.524519 28330 ts_tablet_manager.cc:1939] T 32f9d6979db04f40a5ff1cb262866d35 P 0edc40f3b1034ae9806e1cb201c1c793: Deleting consensus metadata
I20260924 21:17:14.525264 28327 ts_tablet_manager.cc:1905] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:14.526865 28327 ts_tablet_manager.cc:1918] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:14.526942 28327 log.cc:1196] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/32f9d6979db04f40a5ff1cb262866d35
I20260924 21:17:14.527207 28327 ts_tablet_manager.cc:1939] T 32f9d6979db04f40a5ff1cb262866d35 P 7d4cf1aa4744407db2cb3454411b7c00: Deleting consensus metadata
W20260924 21:17:14.528635 27886 connection.cc:535] server connection from 127.0.0.1:35942 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260924 21:17:14.528857 28024 connection.cc:535] server connection from 127.0.0.1:57004 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 21:17:14.536438 28005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.521539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41372 (local address 127.20.115.252:32805)
0924 21:17:14.521579 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.521581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.527460 (+ 5879us) server_negotiation.cc:407] Connection header received
0924 21:17:14.527572 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.527573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.527585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.527616 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:14.528413 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.529210 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.530232 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.530535 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.530576 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.532436 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:14.532437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:14.532437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:14.532445 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.534513 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.534949 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.534951 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.534951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.535009 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.535205 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.535206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.535206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.535751 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.535851 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.536250 (+ 399us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.536327 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:17:14.538146 27673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.530549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42070 (local address 127.20.115.253:41823)
0924 21:17:14.530601 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.530603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.530657 (+ 54us) server_negotiation.cc:407] Connection header received
0924 21:17:14.530806 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.530808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.530826 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.530899 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:14.531679 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.532206 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.533296 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.533614 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.533661 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.535080 (+ 1419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:14.535081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:14.535081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:14.535088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.536912 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.537305 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.537306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.537307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.537366 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.537569 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.537570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.537570 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.537719 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.537821 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.537965 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.538037 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 21:17:14.539263 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:14.530602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41388 (local address 127.20.115.252:32805)
0924 21:17:14.530643 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:14.530645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:14.531331 (+ 686us) server_negotiation.cc:407] Connection header received
0924 21:17:14.531359 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:14.531360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:14.531375 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:14.531450 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:14.532889 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.533621 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.534653 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:14.534992 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:14.535033 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:14.536466 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:14.536468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:14.536468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:14.536475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:14.538232 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.538604 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.538605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.538605 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.538667 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:14.538810 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:14.538810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:14.538811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:14.538939 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:14.539023 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:14.539084 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 21:17:14.539164 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:17:14.540918 28329 tablet_replica.cc:330] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: stopping tablet replica
I20260924 21:17:14.541064 28329 raft_consensus.cc:2229] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:14.541195 28329 raft_consensus.cc:2258] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:14.541709 28329 ts_tablet_manager.cc:1905] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:14.543450 28329 ts_tablet_manager.cc:1918] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:14.543525 28329 log.cc:1196] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/32f9d6979db04f40a5ff1cb262866d35
I20260924 21:17:14.543785 28329 ts_tablet_manager.cc:1939] T 32f9d6979db04f40a5ff1cb262866d35 P 1a7da08f0aa6420597f34f1333393d80: Deleting consensus metadata
W20260924 21:17:14.544281 28160 connection.cc:441] server connection from 127.0.0.1:41078 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41078 (request call id 1) could send its response
I20260924 21:17:15.516928 28338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:15.516547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:57793)
0924 21:17:15.516722 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:15.516873 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
W20260924 21:17:15.517236 28272 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:40423 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:15.518126 28339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:15.517739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.193:37383)
0924 21:17:15.518013 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:15.518080 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":55,"threads_started":1}
W20260924 21:17:15.518291 27999 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:40423 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:15.518399 28340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:15.517759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.194:37291)
0924 21:17:15.518124 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:15.518180 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":67,"threads_started":1}
W20260924 21:17:15.518556 28136 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:40423 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:16.021255 28341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da840c06cc98432690b005cdba3e5e4d)
I20260924 21:17:16.021328 28342 raft_consensus.cc:482] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da840c06cc98432690b005cdba3e5e4d)
I20260924 21:17:16.021421 28341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } }
I20260924 21:17:16.021477 28342 raft_consensus.cc:504] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } }
I20260924 21:17:16.021721 28341 leader_election.cc:288] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423), b85dbe21632b4af8951f35fa0dac492d (127.20.115.253:41823)
I20260924 21:17:16.022073 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.021815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45858)
0924 21:17:16.021950 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.022003 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":45,"threads_started":1}
I20260924 21:17:16.022107 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.021814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45856)
0924 21:17:16.022002 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.022060 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":49,"threads_started":1}
W20260924 21:17:16.022267 27806 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
W20260924 21:17:16.022501 27621 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:16.022681 28342 leader_election.cc:288] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423), 1ad70823ebb04903b0c5b677684c117b (127.20.115.252:32805)
I20260924 21:17:16.023051 28347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.022920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45878)
0924 21:17:16.022984 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.023022 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260924 21:17:16.023121 27621 leader_election.cc:334] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423): Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:16.023728 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.023607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45880)
0924 21:17:16.023668 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.023699 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260924 21:17:16.023821 27806 leader_election.cc:334] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423): Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:16.025168 28348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.022161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41390 (local address 127.20.115.252:32805)
0924 21:17:16.022237 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:16.022239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:16.022247 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:16.022272 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:16.022272 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:16.022283 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:16.022395 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:16.022538 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:16.023284 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:16.024462 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:16.024851 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:16.024920 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:16.024930 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:16.025059 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 21:17:16.025102 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63,"thread_start_us":47,"threads_started":1}
I20260924 21:17:16.025182 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.021904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42074 (local address 127.20.115.253:41823)
0924 21:17:16.022041 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:16.022044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:16.022349 (+ 305us) server_negotiation.cc:407] Connection header received
0924 21:17:16.022371 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:16.022371 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:16.022379 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:16.022441 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:16.022620 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:16.023353 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:16.024411 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:16.024851 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:16.024927 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:16.024936 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:16.025088 (+ 152us) server_negotiation.cc:299] Negotiation successful
0924 21:17:16.025120 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":52,"threads_started":1}
I20260924 21:17:16.025180 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.022002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:32805 (local address 127.0.0.1:41390)
0924 21:17:16.022141 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.022151 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:16.022209 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:16.022401 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:16.022402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:16.022405 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:16.022466 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:16.022468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:16.023360 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:16.023362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:16.024365 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:16.024368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:16.024928 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:16.024928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:16.024986 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:16.024989 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:16.025043 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 21:17:16.025102 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":34,"threads_started":1}
I20260924 21:17:16.025411 27829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b85dbe21632b4af8951f35fa0dac492d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1ad70823ebb04903b0c5b677684c117b" is_pre_election: true
I20260924 21:17:16.025527 27829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b85dbe21632b4af8951f35fa0dac492d in term 1.
I20260924 21:17:16.025168 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.021875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:41823 (local address 127.0.0.1:42074)
0924 21:17:16.022095 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.022106 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:16.022297 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:16.022476 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:16.022477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:16.022478 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:16.022536 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:16.022538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:16.023422 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:16.023423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:16.024334 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:16.024337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:16.024931 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:16.024932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:16.024987 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:16.024990 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:16.025077 (+ 87us) client_negotiation.cc:240] Negotiation successful
0924 21:17:16.025108 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":47,"threads_started":1}
I20260924 21:17:16.025701 27619 leader_election.cc:302] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ad70823ebb04903b0c5b677684c117b, b85dbe21632b4af8951f35fa0dac492d; no voters: da840c06cc98432690b005cdba3e5e4d
I20260924 21:17:16.025815 27644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1ad70823ebb04903b0c5b677684c117b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b85dbe21632b4af8951f35fa0dac492d" is_pre_election: true
I20260924 21:17:16.025931 27644 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ad70823ebb04903b0c5b677684c117b in term 1.
I20260924 21:17:16.026015 28342 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 21:17:16.026074 28342 raft_consensus.cc:482] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Starting leader election (detected failure of leader da840c06cc98432690b005cdba3e5e4d)
I20260924 21:17:16.026088 28342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:17:16.026125 27805 leader_election.cc:302] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ad70823ebb04903b0c5b677684c117b, b85dbe21632b4af8951f35fa0dac492d; no voters: da840c06cc98432690b005cdba3e5e4d
I20260924 21:17:16.026237 28341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260924 21:17:16.026283 28341 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Starting leader election (detected failure of leader da840c06cc98432690b005cdba3e5e4d)
I20260924 21:17:16.026309 28341 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:17:16.026950 28342 raft_consensus.cc:504] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } }
I20260924 21:17:16.027179 28342 leader_election.cc:288] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 2 election: Requested vote from peers da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423), 1ad70823ebb04903b0c5b677684c117b (127.20.115.252:32805)
I20260924 21:17:16.027138 28341 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } }
I20260924 21:17:16.027266 28341 leader_election.cc:288] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [CANDIDATE]: Term 2 election: Requested vote from peers da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423), b85dbe21632b4af8951f35fa0dac492d (127.20.115.253:41823)
I20260924 21:17:16.027246 27829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b85dbe21632b4af8951f35fa0dac492d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1ad70823ebb04903b0c5b677684c117b"
I20260924 21:17:16.027369 27829 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b85dbe21632b4af8951f35fa0dac492d in current term 2: Already voted for candidate 1ad70823ebb04903b0c5b677684c117b in this term.
I20260924 21:17:16.027387 27644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1ad70823ebb04903b0c5b677684c117b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b85dbe21632b4af8951f35fa0dac492d"
I20260924 21:17:16.027456 27644 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1ad70823ebb04903b0c5b677684c117b in current term 2: Already voted for candidate b85dbe21632b4af8951f35fa0dac492d in this term.
I20260924 21:17:16.027508 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.027329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45888)
0924 21:17:16.027457 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.027485 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260924 21:17:16.027575 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.027405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45892)
0924 21:17:16.027512 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.027552 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260924 21:17:16.027830 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.027767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45908)
0924 21:17:16.027797 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.027817 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260924 21:17:16.027905 27621 leader_election.cc:334] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423): Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:16.027935 28343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.027836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:45922)
0924 21:17:16.027886 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.027911 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:17:16.027953 27621 leader_election.cc:302] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b85dbe21632b4af8951f35fa0dac492d; no voters: 1ad70823ebb04903b0c5b677684c117b, da840c06cc98432690b005cdba3e5e4d
I20260924 21:17:16.028012 28342 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260924 21:17:16.028059 27806 leader_election.cc:334] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423): Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:16.028097 27806 leader_election.cc:302] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1ad70823ebb04903b0c5b677684c117b; no voters: b85dbe21632b4af8951f35fa0dac492d, da840c06cc98432690b005cdba3e5e4d
I20260924 21:17:16.028177 28341 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260924 21:17:16.518033 28351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.517745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:49095)
0924 21:17:16.517937 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.517991 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":94,"threads_started":1}
I20260924 21:17:16.518873 28352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.518646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.193:37669)
0924 21:17:16.518775 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.518811 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260924 21:17:16.519024 28353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:16.518794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.194:47765)
0924 21:17:16.518940 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:16.518977 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":61,"threads_started":1}
I20260924 21:17:17.518834 28354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:17.518560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:45855)
0924 21:17:17.518718 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:17.518774 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260924 21:17:17.519464 28355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:17.519272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.193:43679)
0924 21:17:17.519384 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:17.519417 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":52,"threads_started":1}
I20260924 21:17:17.519666 28356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:17.519464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.194:51243)
0924 21:17:17.519593 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:17.519624 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260924 21:17:17.943034 28357 raft_consensus.cc:482] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:17.943135 28357 raft_consensus.cc:504] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } }
I20260924 21:17:17.943318 28357 leader_election.cc:288] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423), 1ad70823ebb04903b0c5b677684c117b (127.20.115.252:32805)
I20260924 21:17:17.943557 27829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b85dbe21632b4af8951f35fa0dac492d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1ad70823ebb04903b0c5b677684c117b" is_pre_election: true
I20260924 21:17:17.943616 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:17.943434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:46188)
0924 21:17:17.943534 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:17.943577 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":62,"threads_started":1}
I20260924 21:17:17.943663 27829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b85dbe21632b4af8951f35fa0dac492d in term 2.
I20260924 21:17:17.943838 27619 leader_election.cc:302] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ad70823ebb04903b0c5b677684c117b, b85dbe21632b4af8951f35fa0dac492d; no voters:
I20260924 21:17:17.943948 28357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260924 21:17:17.943990 28357 raft_consensus.cc:482] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:17.944012 28357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 2 FOLLOWER]: Advancing to term 3
I20260924 21:17:17.944052 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:17.943939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:46202)
0924 21:17:17.943971 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:17.944006 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260924 21:17:17.944142 27621 leader_election.cc:334] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423): Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:17.944857 28357 raft_consensus.cc:504] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } }
I20260924 21:17:17.945013 28357 leader_election.cc:288] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 3 election: Requested vote from peers da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423), 1ad70823ebb04903b0c5b677684c117b (127.20.115.252:32805)
I20260924 21:17:17.945180 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:17.945078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:46212)
0924 21:17:17.945127 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:17.945153 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:17:17.945173 27829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b85dbe21632b4af8951f35fa0dac492d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "1ad70823ebb04903b0c5b677684c117b"
I20260924 21:17:17.945231 27829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 2 FOLLOWER]: Advancing to term 3
I20260924 21:17:17.945498 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:17.945432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:46216)
0924 21:17:17.945463 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:17.945484 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260924 21:17:17.945585 27621 leader_election.cc:334] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423): Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
I20260924 21:17:17.945986 27829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b85dbe21632b4af8951f35fa0dac492d in term 3.
I20260924 21:17:17.946148 27619 leader_election.cc:302] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1ad70823ebb04903b0c5b677684c117b, b85dbe21632b4af8951f35fa0dac492d; no voters: da840c06cc98432690b005cdba3e5e4d
I20260924 21:17:17.946247 28357 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 3 FOLLOWER]: Leader election won for term 3
I20260924 21:17:17.946354 28357 raft_consensus.cc:686] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [term 3 LEADER]: Becoming Leader. State: Replica: b85dbe21632b4af8951f35fa0dac492d, State: Running, Role: LEADER
I20260924 21:17:17.946437 28357 consensus_queue.cc:237] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } }
I20260924 21:17:17.946928 28360 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b85dbe21632b4af8951f35fa0dac492d. Latest consensus state: current_term: 3 leader_uuid: "b85dbe21632b4af8951f35fa0dac492d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:17.947031 28360 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:17.947229 28362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:17.947675 28362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:17.947736 28362 catalog_manager.cc:1259] Loaded cluster ID: 8353c0d9a88f4474ba76971f10fd583e
I20260924 21:17:17.947794 28362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:17.947990 28362 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:17.948092 28362 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Loaded TSK: 0
I20260924 21:17:17.948235 28362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:17:18.028339 27627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:17:18.030208 27627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:18.031363 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.031226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:46224)
0924 21:17:18.031286 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.031325 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 21:17:18.031472 27829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [term 3 FOLLOWER]: Refusing update from remote peer b85dbe21632b4af8951f35fa0dac492d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260924 21:17:18.031819 28360 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260924 21:17:18.031896 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.031698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.0.0.1:46232)
0924 21:17:18.031774 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.031809 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260924 21:17:18.032021 27621 consensus_peers.cc:487] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d -> Peer da840c06cc98432690b005cdba3e5e4d (127.20.115.254:40423): Couldn't send request to peer da840c06cc98432690b005cdba3e5e4d. Status: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260924 21:17:18.032485 28364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b85dbe21632b4af8951f35fa0dac492d. Latest consensus state: current_term: 3 leader_uuid: "b85dbe21632b4af8951f35fa0dac492d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:18.032595 28364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:17:18.033134 28360 sys_catalog.cc:455] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b85dbe21632b4af8951f35fa0dac492d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:18.033241 28360 sys_catalog.cc:458] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:18.033324 28364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b85dbe21632b4af8951f35fa0dac492d" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "da840c06cc98432690b005cdba3e5e4d" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40423 } } peers { permanent_uuid: "b85dbe21632b4af8951f35fa0dac492d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 41823 } attrs { promote: false } } peers { permanent_uuid: "1ad70823ebb04903b0c5b677684c117b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 32805 } attrs { promote: false } } }
I20260924 21:17:18.033468 28364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ad70823ebb04903b0c5b677684c117b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:17:18.033720 28365 mvcc.cc:205] Tried to move back new op lower bound from 7333005877374181376 to 7333005877029228544. Current Snapshot: MvccSnapshot[applied={T|T < 7333005877374181376 or (T in {7333005877374181376})}]
I20260924 21:17:18.038236 28358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.034869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:45431 (local address 127.0.0.1:41080)
0924 21:17:18.034906 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.034920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:18.035100 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:18.035385 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:18.035386 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:18.035387 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:18.035437 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.035439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.036391 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.036392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.037253 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.037256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.038028 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.038029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.038091 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.038095 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:18.038124 (+ 29us) client_negotiation.cc:240] Negotiation successful
0924 21:17:18.038166 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 21:17:18.038391 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.035263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57028 (local address 127.20.115.194:35431)
0924 21:17:18.035374 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:18.035376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:18.035385 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:18.035413 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:18.035414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:18.035425 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:18.035640 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:18.035678 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.036204 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.037539 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.037994 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.038083 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.038093 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:18.038248 (+ 155us) server_negotiation.cc:299] Negotiation successful
0924 21:17:18.038293 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
I20260924 21:17:18.038515 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.035169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41080 (local address 127.20.115.195:45431)
0924 21:17:18.035267 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:18.035268 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:18.035275 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:17:18.035296 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:18.035296 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:18.035304 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:18.035382 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:18.035514 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.036284 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.037386 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.037936 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.038372 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.038383 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:18.038403 (+ 20us) server_negotiation.cc:299] Negotiation successful
0924 21:17:18.038449 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":45,"threads_started":1}
I20260924 21:17:18.038656 28371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.034956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:35431 (local address 127.0.0.1:57028)
0924 21:17:18.035062 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.035072 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:18.035122 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:18.035518 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:18.035519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:18.035522 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:18.035576 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.035578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.036296 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.036297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.037443 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.037445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.038081 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.038082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.038154 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.038158 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:18.038230 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:17:18.038277 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":53,"threads_started":1}
I20260924 21:17:18.038775 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.034917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:42089 (local address 127.0.0.1:35956)
0924 21:17:18.035045 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.035055 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:18.035100 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:18.035662 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:18.035663 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:18.035665 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:18.035697 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.035698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.036560 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.036561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.037634 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.037635 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.038484 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.038485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.038563 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.038566 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:18.038615 (+ 49us) client_negotiation.cc:240] Negotiation successful
0924 21:17:18.038650 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":83,"threads_started":1}
I20260924 21:17:18.038796 28205 tablet_service.cc:1467] Processing CreateTablet for tablet 4d667b305ea84431b92b69c20d646494 (DEFAULT_TABLE table=test-table [id=6ebb8a0bcfc84ce0b0b5d1f3ebcbcb68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:18.038841 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.034999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.20.115.193:42089)
0924 21:17:18.035511 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:18.035513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:18.035521 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:18.035545 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:18.035546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:18.035556 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:18.035617 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:18.035777 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.036482 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.037712 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.038387 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.038448 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.038455 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:18.038715 (+ 260us) server_negotiation.cc:299] Negotiation successful
0924 21:17:18.038760 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":68,"threads_started":1}
I20260924 21:17:18.038936 28205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d667b305ea84431b92b69c20d646494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:18.039143 27932 tablet_service.cc:1467] Processing CreateTablet for tablet 4d667b305ea84431b92b69c20d646494 (DEFAULT_TABLE table=test-table [id=6ebb8a0bcfc84ce0b0b5d1f3ebcbcb68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:18.039284 27932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d667b305ea84431b92b69c20d646494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:18.040426 28375 tablet_bootstrap.cc:492] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: Bootstrap starting.
I20260924 21:17:18.040539 28069 tablet_service.cc:1467] Processing CreateTablet for tablet 4d667b305ea84431b92b69c20d646494 (DEFAULT_TABLE table=test-table [id=6ebb8a0bcfc84ce0b0b5d1f3ebcbcb68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:18.040655 28069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d667b305ea84431b92b69c20d646494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:18.040903 28375 tablet_bootstrap.cc:654] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:18.041501 28375 tablet_bootstrap.cc:492] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: No bootstrap required, opened a new log
I20260924 21:17:18.041549 28375 ts_tablet_manager.cc:1397] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:18.041659 28375 raft_consensus.cc:348] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:18.041718 28375 raft_consensus.cc:374] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:18.041735 28375 raft_consensus.cc:729] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a7da08f0aa6420597f34f1333393d80, State: Initialized, Role: FOLLOWER
I20260924 21:17:18.041783 28375 consensus_queue.cc:260] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:18.041955 28375 ts_tablet_manager.cc:1428] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:18.042181 28379 tablet_bootstrap.cc:492] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: Bootstrap starting.
I20260924 21:17:18.042392 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.042139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:45295)
0924 21:17:18.042278 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.042330 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":98,"threads_started":1}
I20260924 21:17:18.042704 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.042626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:34791)
0924 21:17:18.042660 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.042682 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 21:17:18.042899 28273 heartbeater.cc:503] Master 127.20.115.253:41823 was elected leader, sending a full tablet report...
I20260924 21:17:18.043114 28376 tablet_bootstrap.cc:492] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00: Bootstrap starting.
I20260924 21:17:18.043378 28379 tablet_bootstrap.cc:654] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:18.043490 28376 tablet_bootstrap.cc:654] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:18.043964 28379 tablet_bootstrap.cc:492] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: No bootstrap required, opened a new log
I20260924 21:17:18.044023 28379 ts_tablet_manager.cc:1397] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:18.044135 28379 raft_consensus.cc:348] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:18.044219 28379 raft_consensus.cc:374] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:18.044239 28379 raft_consensus.cc:729] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0edc40f3b1034ae9806e1cb201c1c793, State: Initialized, Role: FOLLOWER
I20260924 21:17:18.044303 28379 consensus_queue.cc:260] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:18.044418 28376 tablet_bootstrap.cc:492] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00: No bootstrap required, opened a new log
I20260924 21:17:18.044472 28376 ts_tablet_manager.cc:1397] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:18.044471 28379 ts_tablet_manager.cc:1428] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:18.044629 28376 raft_consensus.cc:348] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:18.044697 28376 raft_consensus.cc:374] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:18.044716 28376 raft_consensus.cc:729] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d4cf1aa4744407db2cb3454411b7c00, State: Initialized, Role: FOLLOWER
I20260924 21:17:18.044761 28376 consensus_queue.cc:260] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:18.044988 28137 heartbeater.cc:503] Master 127.20.115.253:41823 was elected leader, sending a full tablet report...
I20260924 21:17:18.044989 28376 ts_tablet_manager.cc:1428] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:18.045558 28000 heartbeater.cc:503] Master 127.20.115.253:41823 was elected leader, sending a full tablet report...
I20260924 21:17:18.045991 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.045209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.193:60301)
0924 21:17:18.045857 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.045936 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":74,"threads_started":1}
I20260924 21:17:18.046195 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.045209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.194:46959)
0924 21:17:18.046124 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.046160 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":875,"thread_start_us":71,"threads_started":1}
I20260924 21:17:18.046267 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.046182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.193:46011)
0924 21:17:18.046215 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.046239 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260924 21:17:18.220029 28378 raft_consensus.cc:482] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:18.220177 28378 raft_consensus.cc:504] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:18.220580 28378 leader_election.cc:288] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0edc40f3b1034ae9806e1cb201c1c793 (127.20.115.194:35431), 7d4cf1aa4744407db2cb3454411b7c00 (127.20.115.193:42089)
I20260924 21:17:18.224660 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.220879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:53155 (local address 127.20.115.194:35431)
0924 21:17:18.220935 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:18.220945 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:18.221070 (+ 125us) server_negotiation.cc:407] Connection header received
0924 21:17:18.221186 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:18.221188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:18.221205 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:18.221282 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:18.221401 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.222112 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.223659 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.224221 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.224299 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.224309 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:18.224539 (+ 230us) server_negotiation.cc:299] Negotiation successful
0924 21:17:18.224600 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:17:18.224660 28372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.220853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:42497 (local address 127.20.115.193:42089)
0924 21:17:18.220901 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:18.220903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:18.220914 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:18.220992 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:18.220993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:18.221003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:18.221068 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:18.221266 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.222070 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.223659 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.224221 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.224300 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.224309 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:18.224539 (+ 230us) server_negotiation.cc:299] Negotiation successful
0924 21:17:18.224589 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:17:18.224675 28380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.220764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:42089 (local address 127.20.115.195:42497)
0924 21:17:18.220854 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.220866 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:18.220922 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:18.221121 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:18.221121 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:18.221124 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:18.221186 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.221188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.222181 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.222183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.223560 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.223563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.224320 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.224322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.224413 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.224417 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:18.224493 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:17:18.224546 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":81}
I20260924 21:17:18.224675 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.220815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:35431 (local address 127.20.115.195:53155)
0924 21:17:18.221008 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.221018 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:18.221098 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:18.221302 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:18.221303 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:18.221304 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:18.221338 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.221339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.222183 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.222183 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.223563 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:18.223564 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.224323 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.224324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:18.224413 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.224417 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:18.224488 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 21:17:18.224546 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":115,"thread_start_us":108,"threads_started":1}
I20260924 21:17:18.224972 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d667b305ea84431b92b69c20d646494" candidate_uuid: "1a7da08f0aa6420597f34f1333393d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d4cf1aa4744407db2cb3454411b7c00" is_pre_election: true
I20260924 21:17:18.225071 27952 raft_consensus.cc:2454] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a7da08f0aa6420597f34f1333393d80 in term 0.
I20260924 21:17:18.225250 28158 leader_election.cc:302] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a7da08f0aa6420597f34f1333393d80, 7d4cf1aa4744407db2cb3454411b7c00; no voters:
I20260924 21:17:18.225267 28089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d667b305ea84431b92b69c20d646494" candidate_uuid: "1a7da08f0aa6420597f34f1333393d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0edc40f3b1034ae9806e1cb201c1c793" is_pre_election: true
I20260924 21:17:18.225382 28378 raft_consensus.cc:2781] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:18.225412 28089 raft_consensus.cc:2454] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1a7da08f0aa6420597f34f1333393d80 in term 0.
I20260924 21:17:18.225426 28378 raft_consensus.cc:482] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:18.225450 28378 raft_consensus.cc:3037] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:18.226152 28378 raft_consensus.cc:504] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:18.226296 28378 leader_election.cc:288] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [CANDIDATE]: Term 1 election: Requested vote from peers 0edc40f3b1034ae9806e1cb201c1c793 (127.20.115.194:35431), 7d4cf1aa4744407db2cb3454411b7c00 (127.20.115.193:42089)
I20260924 21:17:18.226438 28089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d667b305ea84431b92b69c20d646494" candidate_uuid: "1a7da08f0aa6420597f34f1333393d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0edc40f3b1034ae9806e1cb201c1c793"
I20260924 21:17:18.226514 28089 raft_consensus.cc:3037] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:18.226513 27952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d667b305ea84431b92b69c20d646494" candidate_uuid: "1a7da08f0aa6420597f34f1333393d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d4cf1aa4744407db2cb3454411b7c00"
I20260924 21:17:18.226590 27952 raft_consensus.cc:3037] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:18.227161 27952 raft_consensus.cc:2454] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a7da08f0aa6420597f34f1333393d80 in term 1.
I20260924 21:17:18.227202 28089 raft_consensus.cc:2454] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a7da08f0aa6420597f34f1333393d80 in term 1.
I20260924 21:17:18.227360 28158 leader_election.cc:302] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a7da08f0aa6420597f34f1333393d80, 7d4cf1aa4744407db2cb3454411b7c00; no voters:
I20260924 21:17:18.227452 28378 raft_consensus.cc:2781] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:18.227596 28378 raft_consensus.cc:686] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 1 LEADER]: Becoming Leader. State: Replica: 1a7da08f0aa6420597f34f1333393d80, State: Running, Role: LEADER
I20260924 21:17:18.227650 28378 consensus_queue.cc:237] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } }
I20260924 21:17:18.228086 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.227981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:57959)
0924 21:17:18.228020 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.228058 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 21:17:18.228618 27633 catalog_manager.cc:5495] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a7da08f0aa6420597f34f1333393d80 (127.20.115.195). New cstate: current_term: 1 leader_uuid: "1a7da08f0aa6420597f34f1333393d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a7da08f0aa6420597f34f1333393d80" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 } health_report { overall_health: UNKNOWN } } }
I20260924 21:17:18.266765 28373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.263765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41092 (local address 127.20.115.195:45431)
0924 21:17:18.263858 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:18.263860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:18.263868 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:18.263904 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:18.263905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:18.263922 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:18.263985 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:18.264343 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.264882 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.265881 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:18.266170 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:18.266228 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:18.266424 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:18.266527 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:18.266610 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:18.266630 (+ 20us) server_negotiation.cc:299] Negotiation successful
0924 21:17:18.266658 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 21:17:18.268154 27952 raft_consensus.cc:1261] T 4d667b305ea84431b92b69c20d646494 P 7d4cf1aa4744407db2cb3454411b7c00 [term 1 FOLLOWER]: Refusing update from remote peer 1a7da08f0aa6420597f34f1333393d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:18.268175 28089 raft_consensus.cc:1261] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [term 1 FOLLOWER]: Refusing update from remote peer 1a7da08f0aa6420597f34f1333393d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:18.268463 28378 consensus_queue.cc:1035] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d4cf1aa4744407db2cb3454411b7c00" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 42089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:18.268611 28388 consensus_queue.cc:1035] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0edc40f3b1034ae9806e1cb201c1c793" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:18.269537 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.269402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.193:42965)
0924 21:17:18.269446 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.269497 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260924 21:17:18.269737 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.269533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.194:49435)
0924 21:17:18.269674 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.269703 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260924 21:17:18.270350 28391 mvcc.cc:205] Tried to move back new op lower bound from 7333005878343749632 to 7333005878181150720. Current Snapshot: MvccSnapshot[applied={T|T < 7333005878343749632}]
I20260924 21:17:18.271446 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.270671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:47731)
0924 21:17:18.271387 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.271419 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":700}
I20260924 21:17:18.271468 28385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.271357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.194:51567)
0924 21:17:18.271401 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.271427 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260924 21:17:18.271761 28386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.271670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.193:53707)
0924 21:17:18.271706 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.271730 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260924 21:17:18.271842 28387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.271763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40423 (local address 127.20.115.195:35489)
0924 21:17:18.271795 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.271817 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:40423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260924 21:17:18.273954 27627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:18.274061 27627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:18.275388 27627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b85dbe21632b4af8951f35fa0dac492d: Sending DeleteTablet for 3 replicas of tablet 4d667b305ea84431b92b69c20d646494
I20260924 21:17:18.275686 28205 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d667b305ea84431b92b69c20d646494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:18 UTC) from {username='oryx', principal='oryx/127.20.115.253@KRBTEST.COM'} at 127.0.0.1:41080
I20260924 21:17:18.275751 28069 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d667b305ea84431b92b69c20d646494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:18 UTC) from {username='oryx', principal='oryx/127.20.115.253@KRBTEST.COM'} at 127.0.0.1:57028
I20260924 21:17:18.276014 28400 tablet_replica.cc:330] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: stopping tablet replica
I20260924 21:17:18.276054 27932 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d667b305ea84431b92b69c20d646494 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:18 UTC) from {username='oryx', principal='oryx/127.20.115.253@KRBTEST.COM'} at 127.0.0.1:35956
I20260924 21:17:18.276146 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 27781
I20260924 21:17:18.276988 28400 raft_consensus.cc:2229] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:17:18.277062 28401 tablet_replica.cc:330] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: stopping tablet replica
I20260924 21:17:18.277158 28401 raft_consensus.cc:2229] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:18.277176 28400 raft_consensus.cc:2258] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:18.277228 28401 raft_consensus.cc:2258] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:18.277518 28400 ts_tablet_manager.cc:1905] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:18.279196 28400 ts_tablet_manager.cc:1918] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:18.279265 28400 log.cc:1196] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/4d667b305ea84431b92b69c20d646494
I20260924 21:17:18.279521 28400 ts_tablet_manager.cc:1939] T 4d667b305ea84431b92b69c20d646494 P 1a7da08f0aa6420597f34f1333393d80: Deleting consensus metadata
I20260924 21:17:18.279844 27620 catalog_manager.cc:4841] TS 1a7da08f0aa6420597f34f1333393d80 (127.20.115.195:45431): tablet 4d667b305ea84431b92b69c20d646494 (table test-table [id=6ebb8a0bcfc84ce0b0b5d1f3ebcbcb68]) successfully deleted
I20260924 21:17:18.280961 28401 ts_tablet_manager.cc:1905] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:18.282389 28401 ts_tablet_manager.cc:1918] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:18.282508 28401 log.cc:1196] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/4d667b305ea84431b92b69c20d646494
I20260924 21:17:18.282572 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 28006
I20260924 21:17:18.282807 28401 ts_tablet_manager.cc:1939] T 4d667b305ea84431b92b69c20d646494 P 0edc40f3b1034ae9806e1cb201c1c793: Deleting consensus metadata
I20260924 21:17:18.283139 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.282677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:42089 (local address 127.0.0.1:35972)
0924 21:17:18.283071 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.283102 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:42089: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376}
W20260924 21:17:18.283294 27618 catalog_manager.cc:4584] TS 7d4cf1aa4744407db2cb3454411b7c00 (127.20.115.193:42089): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4d667b305ea84431b92b69c20d646494: Network error: Client connection negotiation failed: client connection to 127.20.115.193:42089: connect: Connection refused (error 111)
I20260924 21:17:18.289986 28370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:18.289836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:35431 (local address 127.0.0.1:57034)
0924 21:17:18.289898 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:18.289925 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.194:35431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 21:17:18.289978 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 28142
I20260924 21:17:18.294919 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 27499
I20260924 21:17:18.300004 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 27688
2026-09-24T21:17:18Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6905 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:18 dist-test-slave-cxpd krb5kdc[28405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:18 dist-test-slave-cxpd krb5kdc[28405](info): set up 2 sockets
Sep 24 21:17:18 dist-test-slave-cxpd krb5kdc[28405](info): commencing operation
krb5kdc: starting...
W20260924 21:17:19.333195 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:19 dist-test-slave-cxpd krb5kdc[28405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:19Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.254.keytab.
Entry for principal oryx/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.254.keytab.
I20260924 21:17:19.452332 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:41495
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.254.keytab
--principal=oryx/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:41495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:19.532948 28418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:19.533139 28418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:19.533192 28418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:19.534886 28418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:19.534929 28418 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260924 21:17:19.534943 28418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:19.534956 28418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:19.534974 28418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:19.534987 28418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:19.536489 28418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:41495
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.254.keytab
--principal=oryx/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:41495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:19.536679 28418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:19.536904 28418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:17:19.541992 28418 server_base.cc:1034] running on GCE node
W20260924 21:17:19.541976 28423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:19.542076 28424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:19.541954 28426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:19.542383 28418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:19.542706 28418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:19.543898 28418 hybrid_clock.cc:648] HybridClock initialized: now 1790284639543887 us; error 41 us; skew 500 ppm
Sep 24 21:17:19 dist-test-slave-cxpd krb5kdc[28405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284639, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:19.546384 28418 init.cc:377] Logged in from keytab as oryx/127.20.115.254@KRBTEST.COM (short username oryx)
I20260924 21:17:19.546893 28418 webserver.cc:460] Webserver started at http://127.20.115.254:34313/ using document root <none> and password file <none>
I20260924 21:17:19.547122 28418 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:19.547192 28418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:19.547340 28418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:19.548398 28418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "22f8cba9686b474b97f115f9873d083a"
format_stamp: "Formatted at 2026-09-24 21:17:19 on dist-test-slave-cxpd"
server_key: "bd4bb4c13eb377613133c35ff49cc3e4"
server_key_iv: "ffe2e3d28c55815905a1bc8ea82155c3"
server_key_version: "encryptionkey@0"
I20260924 21:17:19.548743 28418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "22f8cba9686b474b97f115f9873d083a"
format_stamp: "Formatted at 2026-09-24 21:17:19 on dist-test-slave-cxpd"
server_key: "bd4bb4c13eb377613133c35ff49cc3e4"
server_key_iv: "ffe2e3d28c55815905a1bc8ea82155c3"
server_key_version: "encryptionkey@0"
I20260924 21:17:19.550173 28418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:19.550930 28433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:19.551139 28418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:19.551239 28418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "22f8cba9686b474b97f115f9873d083a"
format_stamp: "Formatted at 2026-09-24 21:17:19 on dist-test-slave-cxpd"
server_key: "bd4bb4c13eb377613133c35ff49cc3e4"
server_key_iv: "ffe2e3d28c55815905a1bc8ea82155c3"
server_key_version: "encryptionkey@0"
I20260924 21:17:19.551321 28418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:19.565209 28418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:19.567071 28418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:19.567235 28418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:19.572044 28418 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:41495
I20260924 21:17:19.572086 28495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:41495 every 8 connection(s)
I20260924 21:17:19.572391 28418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:19.573084 28496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:19.575214 28496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a: Bootstrap starting.
I20260924 21:17:19.575752 28496 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:19.575984 28496 log.cc:824] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:19.576614 28496 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a: No bootstrap required, opened a new log
I20260924 21:17:19.577023 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 28418
I20260924 21:17:19.577131 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:19.577337 20943 external_mini_cluster.cc:1428] Setting key 97619eeb14995d4b1b19e975deb6e9ce
I20260924 21:17:19.579025 28496 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f8cba9686b474b97f115f9873d083a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41495 } }
I20260924 21:17:19.579190 28496 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:19.579231 28496 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f8cba9686b474b97f115f9873d083a, State: Initialized, Role: FOLLOWER
I20260924 21:17:19.579413 28496 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f8cba9686b474b97f115f9873d083a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41495 } }
I20260924 21:17:19.579507 28496 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:19.579540 28496 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:19.579592 28496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:19.580225 28496 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f8cba9686b474b97f115f9873d083a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41495 } }
I20260924 21:17:19.580309 28496 leader_election.cc:302] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22f8cba9686b474b97f115f9873d083a; no voters:
I20260924 21:17:19.580478 28496 leader_election.cc:288] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:19.580523 28501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:19.580658 28501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [term 1 LEADER]: Becoming Leader. State: Replica: 22f8cba9686b474b97f115f9873d083a, State: Running, Role: LEADER
I20260924 21:17:19.580735 28496 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:19.580809 28501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f8cba9686b474b97f115f9873d083a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41495 } }
I20260924 21:17:19.581138 28501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22f8cba9686b474b97f115f9873d083a. Latest consensus state: current_term: 1 leader_uuid: "22f8cba9686b474b97f115f9873d083a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f8cba9686b474b97f115f9873d083a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41495 } } }
I20260924 21:17:19.581249 28501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:19.581151 28502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22f8cba9686b474b97f115f9873d083a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f8cba9686b474b97f115f9873d083a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 41495 } } }
I20260924 21:17:19.581408 28502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:19.581579 28508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:19.582085 28508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:19.584717 28508 catalog_manager.cc:1347] Generated new cluster ID: 1448c58d7bb14721bbd11bf8ec1512de
I20260924 21:17:19.584796 28508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:17:19 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:19.589864 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:19.577872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39268 (local address 127.20.115.254:41495)
0924 21:17:19.578049 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:19.578056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:19.578081 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:17:19.579012 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:19.579018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:19.579065 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:19.579177 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:19.580008 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.581304 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.582468 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.583057 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.583190 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:19.585748 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:19.585752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:19.585752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:19.585764 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:19.588269 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.588654 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.588658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.588658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.588773 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.588988 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.588989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.588989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.589297 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:19.589462 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:19.589532 (+ 70us) server_negotiation.cc:299] Negotiation successful
0924 21:17:19.589668 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260924 21:17:19.604537 28508 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:19.605155 28508 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:19.611084 28508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a: Generated new TSK 0
I20260924 21:17:19.611274 28508 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab.
Entry for principal oryx/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab.
I20260924 21:17:19.639834 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:41495
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab
--principal=oryx/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:19.720819 28522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:19.721026 28522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:19.721050 28522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:19.722213 28522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:19.722255 28522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:19.722276 28522 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:17:19.722335 28522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:19.723801 28522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab
--principal=oryx/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.115.254:41495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:19.724112 28522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:19.724342 28522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:19.729521 28530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:19.729564 28527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:19.729604 28522 server_base.cc:1034] running on GCE node
W20260924 21:17:19.729524 28528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:19.729974 28522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:19.730242 28522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:19.731403 28522 hybrid_clock.cc:648] HybridClock initialized: now 1790284639731385 us; error 39 us; skew 500 ppm
Sep 24 21:17:19 dist-test-slave-cxpd krb5kdc[28405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284639, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:19.733945 28522 init.cc:377] Logged in from keytab as oryx/127.20.115.193@KRBTEST.COM (short username oryx)
I20260924 21:17:19.734373 28522 webserver.cc:460] Webserver started at http://127.20.115.193:42955/ using document root <none> and password file <none>
I20260924 21:17:19.734594 28522 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:19.734661 28522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:19.734824 28522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:19.735867 28522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "09cbcdb8a4764fcaa54c61e797ff6d40"
format_stamp: "Formatted at 2026-09-24 21:17:19 on dist-test-slave-cxpd"
server_key: "b064e1d70324b4f2f5d730352045ff86"
server_key_iv: "c9667a522ac19554d367b3ebfbf0ea18"
server_key_version: "encryptionkey@0"
I20260924 21:17:19.736243 28522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "09cbcdb8a4764fcaa54c61e797ff6d40"
format_stamp: "Formatted at 2026-09-24 21:17:19 on dist-test-slave-cxpd"
server_key: "b064e1d70324b4f2f5d730352045ff86"
server_key_iv: "c9667a522ac19554d367b3ebfbf0ea18"
server_key_version: "encryptionkey@0"
I20260924 21:17:19.737675 28522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:17:19.738430 28537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:19.738628 28522 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:19.738725 28522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "09cbcdb8a4764fcaa54c61e797ff6d40"
format_stamp: "Formatted at 2026-09-24 21:17:19 on dist-test-slave-cxpd"
server_key: "b064e1d70324b4f2f5d730352045ff86"
server_key_iv: "c9667a522ac19554d367b3ebfbf0ea18"
server_key_version: "encryptionkey@0"
I20260924 21:17:19.738806 28522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:19.758247 28522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:19.760155 28522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:19.760311 28522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:19.760892 28522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:19.760934 28522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:19.760957 28522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:19.760970 28522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:19.767874 28522 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:36901
I20260924 21:17:19.767967 28652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:36901 every 8 connection(s)
Sep 24 21:17:19 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284639, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:19.768280 28522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:17:19.771660 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:19.761347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:44959 (local address 127.20.115.254:41495)
0924 21:17:19.761426 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:19.761429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:19.761620 (+ 191us) server_negotiation.cc:407] Connection header received
0924 21:17:19.762131 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:19.762133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:19.762148 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:19.762226 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:19.763140 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.763989 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.765714 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.766035 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.766090 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:19.768902 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:19.768904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:19.768904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:19.768913 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:19.770412 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.770835 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.770836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.770836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.770895 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.771149 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.771150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.771150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.771258 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:19.771312 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:19.771526 (+ 214us) server_negotiation.cc:299] Negotiation successful
0924 21:17:19.771597 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 21:17:19.771834 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:19.761310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41495 (local address 127.20.115.193:44959)
0924 21:17:19.761479 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:19.761522 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:19.762025 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:19.762331 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:19.762337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:19.762729 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:19.763052 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:19.763058 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.764113 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.764114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:19.765596 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:19.765599 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.766121 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.766122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:19.766294 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:19.766852 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:19.766863 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:19.768716 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:19.770502 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:19.770509 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:19.770510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:19.770741 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:19.770985 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:19.770986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:19.770986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:19.771090 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:19.771314 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:19.771318 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:19.771429 (+ 111us) client_negotiation.cc:769] Sending connection context
0924 21:17:19.771514 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 21:17:19.771633 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":101,"threads_started":1}
I20260924 21:17:19.772384 28653 heartbeater.cc:347] Connected to a master server at 127.20.115.254:41495
I20260924 21:17:19.772495 28653 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:19.772692 28653 heartbeater.cc:511] Master 127.20.115.254:41495 requested a full tablet report, sending...
I20260924 21:17:19.773299 28450 ts_manager.cc:194] Registered new tserver with Master: 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901)
I20260924 21:17:19.774219 28450 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:44959
I20260924 21:17:19.774657 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 28522
I20260924 21:17:19.774771 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:19.775017 20943 external_mini_cluster.cc:1428] Setting key 9a4ecbfd290e9ed8dffd1a1f0a6fd5ac
I20260924 21:17:19.779318 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:19.773419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41495 (local address 127.20.115.193:49893)
0924 21:17:19.773564 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:19.773575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:19.773632 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:19.773750 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:19.773751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:19.773939 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:19.774002 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:19.774003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.774824 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.774825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:19.775630 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:19.775633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.775968 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.775969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:19.776045 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:19.776475 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:19.776480 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:19.776925 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:19.778392 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:19.778392 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:19.778393 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:19.778564 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:19.778772 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:19.778773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:19.778773 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:19.778823 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:19.779093 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:19.779093 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:19.779128 (+ 35us) client_negotiation.cc:769] Sending connection context
0924 21:17:19.779178 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:17:19.779253 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260924 21:17:19.779421 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:19.773476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:49893 (local address 127.20.115.254:41495)
0924 21:17:19.773516 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:19.773518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:19.773635 (+ 117us) server_negotiation.cc:407] Connection header received
0924 21:17:19.773687 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:19.773688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:19.773695 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:19.773737 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:19.774062 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.774689 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.775712 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.775891 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.775946 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:19.777035 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:19.777037 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:19.777038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:19.777045 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:19.778323 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.778644 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.778645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.778645 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.778704 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.778885 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.778886 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.778886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.779033 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:19.779099 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:19.779201 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 21:17:19.779289 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
WARNING: no policy specified for oryx/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab.
Entry for principal oryx/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab.
I20260924 21:17:19.800750 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:41495
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab
--principal=oryx/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:19.880294 28662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:19.880559 28662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:19.880585 28662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:19.881868 28662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:19.881912 28662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:19.881932 28662 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:17:19.881950 28662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:19.883451 28662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab
--principal=oryx/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.115.254:41495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:19.883635 28662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:19.883805 28662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:19.888882 28667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:19.888854 28670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:19.888895 28668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:19.889045 28662 server_base.cc:1034] running on GCE node
I20260924 21:17:19.889340 28662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:19.889559 28662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:19.890697 28662 hybrid_clock.cc:648] HybridClock initialized: now 1790284639890680 us; error 37 us; skew 500 ppm
Sep 24 21:17:19 dist-test-slave-cxpd krb5kdc[28405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284639, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:19.893184 28662 init.cc:377] Logged in from keytab as oryx/127.20.115.194@KRBTEST.COM (short username oryx)
I20260924 21:17:19.893667 28662 webserver.cc:460] Webserver started at http://127.20.115.194:46437/ using document root <none> and password file <none>
I20260924 21:17:19.893899 28662 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:19.893968 28662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:19.894122 28662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:19.895241 28662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "c37f18682b274a818393b757f49f4a51"
format_stamp: "Formatted at 2026-09-24 21:17:19 on dist-test-slave-cxpd"
server_key: "8a85bb030249983939a15a0368c62a98"
server_key_iv: "65f4d57d4587a8eeb2cc018607614338"
server_key_version: "encryptionkey@0"
I20260924 21:17:19.895603 28662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "c37f18682b274a818393b757f49f4a51"
format_stamp: "Formatted at 2026-09-24 21:17:19 on dist-test-slave-cxpd"
server_key: "8a85bb030249983939a15a0368c62a98"
server_key_iv: "65f4d57d4587a8eeb2cc018607614338"
server_key_version: "encryptionkey@0"
I20260924 21:17:19.897032 28662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:19.897756 28677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:19.897928 28662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:19.898036 28662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "c37f18682b274a818393b757f49f4a51"
format_stamp: "Formatted at 2026-09-24 21:17:19 on dist-test-slave-cxpd"
server_key: "8a85bb030249983939a15a0368c62a98"
server_key_iv: "65f4d57d4587a8eeb2cc018607614338"
server_key_version: "encryptionkey@0"
I20260924 21:17:19.898109 28662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:19.913367 28662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:19.915285 28662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:19.915445 28662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:19.915997 28662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:19.916055 28662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:19.916133 28662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:19.916185 28662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:17:19 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284639, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:19.923553 28662 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:40623
I20260924 21:17:19.923609 28792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:40623 every 8 connection(s)
I20260924 21:17:19.923907 28662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:17:19.925272 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 28662
I20260924 21:17:19.925397 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:19.925629 20943 external_mini_cluster.cc:1428] Setting key a0af91292863b213138b702942ec00b2
I20260924 21:17:19.926647 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:19.916377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:39735 (local address 127.20.115.254:41495)
0924 21:17:19.916422 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:19.916424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:19.916689 (+ 265us) server_negotiation.cc:407] Connection header received
0924 21:17:19.917446 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:19.917447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:19.917458 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:19.917521 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:19.918451 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.918940 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.920621 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.920943 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.921022 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:19.923331 (+ 2309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:19.923333 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:19.923333 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:19.923341 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:19.925090 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.925591 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.925592 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.925592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.925644 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.925991 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.925992 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.925993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.926128 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:19.926201 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:19.926521 (+ 320us) server_negotiation.cc:299] Negotiation successful
0924 21:17:19.926584 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:17:19.926934 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:19.916379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41495 (local address 127.20.115.194:39735)
0924 21:17:19.916532 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:19.916578 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:19.917338 (+ 760us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:19.917606 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:19.917611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:19.918031 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:19.918338 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:19.918346 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.919017 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.919018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:19.920495 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:19.920499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.921043 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.921044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:19.921172 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:19.921690 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:19.921697 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:19.923223 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:19.925174 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:19.925179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:19.925181 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:19.925517 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:19.925737 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:19.925738 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:19.925738 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:19.925924 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:19.926191 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:19.926195 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:19.926365 (+ 170us) client_negotiation.cc:769] Sending connection context
0924 21:17:19.926489 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 21:17:19.926669 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260924 21:17:19.927416 28793 heartbeater.cc:347] Connected to a master server at 127.20.115.254:41495
I20260924 21:17:19.927531 28793 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:19.927738 28793 heartbeater.cc:511] Master 127.20.115.254:41495 requested a full tablet report, sending...
I20260924 21:17:19.928156 28450 ts_manager.cc:194] Registered new tserver with Master: c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623)
I20260924 21:17:19.928673 28450 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:39735
WARNING: no policy specified for oryx/127.20.115.195@KRBTEST.COM; defaulting to no policy
I20260924 21:17:19.934170 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:19.928476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:43011 (local address 127.20.115.254:41495)
0924 21:17:19.928521 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:19.928523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:19.928674 (+ 151us) server_negotiation.cc:407] Connection header received
0924 21:17:19.928707 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:19.928707 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:19.928718 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:19.928795 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:19.929224 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.929775 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.930543 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.930744 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.930815 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:19.931695 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:19.931696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:19.931696 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:19.931702 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:19.933023 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.933365 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.933366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.933367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.933422 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:19.933628 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:19.933629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:19.933629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:19.933768 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:19.933846 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:19.934011 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 21:17:19.934083 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:17:19.934185 28800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:19.928415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41495 (local address 127.20.115.194:43011)
0924 21:17:19.928589 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:19.928601 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:19.928668 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:19.928836 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:19.928837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:19.929075 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:19.929131 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:19.929132 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.929870 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.929871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:19.930445 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:19.930447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:19.930825 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:19.930826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:19.930888 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:19.931326 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:19.931330 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:19.931596 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:19.933112 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:19.933113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:19.933113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:19.933282 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:19.933501 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:19.933502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:19.933502 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:19.933558 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:19.933838 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:19.933839 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:19.933883 (+ 44us) client_negotiation.cc:769] Sending connection context
0924 21:17:19.933984 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 21:17:19.934084 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab.
Entry for principal oryx/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab.
I20260924 21:17:19.951545 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:41495
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab
--principal=oryx/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:20.032658 28802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:20.032914 28802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:20.032963 28802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:20.034130 28802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:20.034196 28802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:20.034233 28802 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260924 21:17:20.034291 28802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:20.035789 28802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab
--principal=oryx/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.115.254:41495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:20.036058 28802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:20.036283 28802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:20.041265 28810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:20.041308 28808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:20.041266 28807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:20.041564 28802 server_base.cc:1034] running on GCE node
I20260924 21:17:20.041785 28802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:20.042058 28802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:20.043220 28802 hybrid_clock.cc:648] HybridClock initialized: now 1790284640043196 us; error 37 us; skew 500 ppm
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284640, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:20.045789 28802 init.cc:377] Logged in from keytab as oryx/127.20.115.195@KRBTEST.COM (short username oryx)
I20260924 21:17:20.046245 28802 webserver.cc:460] Webserver started at http://127.20.115.195:34463/ using document root <none> and password file <none>
I20260924 21:17:20.046484 28802 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:20.046551 28802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:20.046700 28802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:20.047720 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2"
format_stamp: "Formatted at 2026-09-24 21:17:20 on dist-test-slave-cxpd"
server_key: "83d7a658bbd78b614a1e933433f2f484"
server_key_iv: "285d7af91579b6381e200c09cf4fdde2"
server_key_version: "encryptionkey@0"
I20260924 21:17:20.048087 28802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2"
format_stamp: "Formatted at 2026-09-24 21:17:20 on dist-test-slave-cxpd"
server_key: "83d7a658bbd78b614a1e933433f2f484"
server_key_iv: "285d7af91579b6381e200c09cf4fdde2"
server_key_version: "encryptionkey@0"
I20260924 21:17:20.049525 28802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:20.050277 28817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.050444 28802 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:20.050551 28802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2"
format_stamp: "Formatted at 2026-09-24 21:17:20 on dist-test-slave-cxpd"
server_key: "83d7a658bbd78b614a1e933433f2f484"
server_key_iv: "285d7af91579b6381e200c09cf4fdde2"
server_key_version: "encryptionkey@0"
I20260924 21:17:20.050623 28802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:20.059077 28802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:20.061363 28802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:20.061522 28802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:20.061988 28802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:20.062053 28802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.062098 28802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:20.062125 28802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284640, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:20.069382 28802 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:38831
I20260924 21:17:20.069449 28932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:38831 every 8 connection(s)
I20260924 21:17:20.069823 28802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:17:20.071985 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.062455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:36499 (local address 127.20.115.254:41495)
0924 21:17:20.062528 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.062531 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.062708 (+ 177us) server_negotiation.cc:407] Connection header received
0924 21:17:20.063189 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.063191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.063205 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.063294 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:20.064169 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.065033 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.066019 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.066350 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.066430 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.068663 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:20.068664 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:20.068665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:20.068673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:20.070400 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.070757 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.070758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.070758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.070810 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.071124 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.071125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.071125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.071307 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:20.071377 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.071747 (+ 370us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.071861 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:17:20.072130 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.062423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41495 (local address 127.20.115.195:36499)
0924 21:17:20.062578 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.062618 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.063092 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.063383 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.063387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.063763 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:20.064059 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.064068 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.065137 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.065138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.065952 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.065954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.066452 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.066454 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.066611 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.067136 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:20.067158 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:20.068588 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:20.070479 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:20.070483 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:20.070484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:20.070684 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:20.070913 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:20.070913 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:20.070914 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:20.071064 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:20.071369 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:20.071375 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:20.071570 (+ 195us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.071718 (+ 148us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.071898 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":89,"threads_started":1}
I20260924 21:17:20.072598 28933 heartbeater.cc:347] Connected to a master server at 127.20.115.254:41495
I20260924 21:17:20.072703 28933 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:20.072940 28933 heartbeater.cc:511] Master 127.20.115.254:41495 requested a full tablet report, sending...
I20260924 21:17:20.073403 28449 ts_manager.cc:194] Registered new tserver with Master: c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831)
I20260924 21:17:20.073860 28449 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:36499
I20260924 21:17:20.076529 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 28802
I20260924 21:17:20.076647 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:20.076864 20943 external_mini_cluster.cc:1428] Setting key a9fd8c7291fda14b6034b91e19d8deae
I20260924 21:17:20.077670 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:17:20.079288 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.073689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:44781 (local address 127.20.115.254:41495)
0924 21:17:20.073733 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.073735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.073849 (+ 114us) server_negotiation.cc:407] Connection header received
0924 21:17:20.073924 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.073924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.073935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.073965 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:20.074361 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.074825 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.075599 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.075770 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.075819 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.076724 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:20.076726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:20.076726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:20.076733 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:20.078198 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.078585 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.078586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.078586 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.078633 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.078833 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.078834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.078834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.078947 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:20.079013 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.079173 (+ 160us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.079229 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:17:20.079417 28939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.073615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41495 (local address 127.20.115.195:44781)
0924 21:17:20.073780 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.073790 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.073864 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.074006 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.074007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.074214 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:20.074283 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.074285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.074995 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.074996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.075526 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.075528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.075850 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.075851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.075909 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.076278 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:20.076282 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:20.076544 (+ 262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:20.078277 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:20.078279 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:20.078279 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:20.078510 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:20.078677 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:20.078678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:20.078678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:20.078743 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:20.079024 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:20.079025 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:20.079075 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.079155 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.079269 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284640, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 21:17:20.100143 28500 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:20.094448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39276 (local address 127.20.115.254:41495)
0924 21:17:20.094519 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.094521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.094531 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:20.094586 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.094587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.094596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.094646 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:20.095198 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.095692 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.096418 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.096624 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.096691 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.100077 (+ 3386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39276: BlockingRecv error: recv got EOF from 127.0.0.1:39276 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284640, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:20.107751 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.101005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39282 (local address 127.20.115.254:41495)
0924 21:17:20.101073 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.101075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.101160 (+ 85us) server_negotiation.cc:407] Connection header received
0924 21:17:20.101242 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.101243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.101254 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.101304 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:20.101783 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.102339 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.103022 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.103221 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.103295 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.105444 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:20.105445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:20.105445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:20.105451 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:20.106671 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.107053 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.107054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.107055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.107110 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.107284 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.107284 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.107285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.107396 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:20.107475 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.107615 (+ 140us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.107677 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:17:20.109369 28449 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:17:20.111091 28449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:20.117874 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.113965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.20.115.194:40623)
0924 21:17:20.114407 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.114412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.114421 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:20.114460 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.114462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.114477 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.114704 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:20.114759 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.115811 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.116951 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.117461 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.117573 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.117640 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.117733 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.117771 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":80,"threads_started":1}
I20260924 21:17:20.117864 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.113734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40623 (local address 127.0.0.1:48882)
0924 21:17:20.114046 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.114057 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.114099 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.114586 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.114587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.114590 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:20.114633 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.114634 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.115901 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.115902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.116876 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.116879 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.117551 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.117552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.117631 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.117644 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.117719 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.117772 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":51,"threads_started":1}
I20260924 21:17:20.118117 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.113711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:38831 (local address 127.0.0.1:50688)
0924 21:17:20.113830 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.113873 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.113944 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.114226 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.114228 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.114230 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:20.114328 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.114330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.115767 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.115768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.116890 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.116892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.117772 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.117773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.117837 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.117841 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.117872 (+ 31us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.117915 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":22,"thread_start_us":47,"threads_started":1}
I20260924 21:17:20.118117 28957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.113768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:36901 (local address 127.0.0.1:58350)
0924 21:17:20.114049 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.114057 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.114099 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.114561 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.114562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.114563 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:20.114602 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.114603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.115479 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.115480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.116800 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.116802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.117852 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.117853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.117917 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.117920 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.117983 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.118027 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":28,"thread_start_us":64,"threads_started":1}
I20260924 21:17:20.118324 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.114153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.20.115.193:36901)
0924 21:17:20.114336 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.114341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.114351 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:20.114388 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.114391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.114407 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.114494 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:20.114680 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.115374 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.116898 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.117666 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.117765 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.117821 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.118154 (+ 333us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.118206 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":84,"threads_started":1}
I20260924 21:17:20.118638 28727 tablet_service.cc:1467] Processing CreateTablet for tablet f4d7b7e78f8f46ea82914827f1cb56e5 (DEFAULT_TABLE table=test-table [id=a4449213502646c8bc6640f7c88883c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.118858 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.113861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50688 (local address 127.20.115.195:38831)
0924 21:17:20.114043 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.114049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.114061 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:17:20.114101 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.114103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.114120 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.114214 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:20.114459 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.115682 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.116974 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.117680 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.118678 (+ 998us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.118739 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.118760 (+ 21us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.118792 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260924 21:17:20.119024 28727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d7b7e78f8f46ea82914827f1cb56e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.118968 28587 tablet_service.cc:1467] Processing CreateTablet for tablet f4d7b7e78f8f46ea82914827f1cb56e5 (DEFAULT_TABLE table=test-table [id=a4449213502646c8bc6640f7c88883c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.119262 28587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d7b7e78f8f46ea82914827f1cb56e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.119372 28867 tablet_service.cc:1467] Processing CreateTablet for tablet f4d7b7e78f8f46ea82914827f1cb56e5 (DEFAULT_TABLE table=test-table [id=a4449213502646c8bc6640f7c88883c5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.119623 28867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d7b7e78f8f46ea82914827f1cb56e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.122491 28961 tablet_bootstrap.cc:492] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Bootstrap starting.
I20260924 21:17:20.123050 28962 tablet_bootstrap.cc:492] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: Bootstrap starting.
I20260924 21:17:20.123143 28961 tablet_bootstrap.cc:654] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.123471 28961 log.cc:824] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:20.123680 28962 tablet_bootstrap.cc:654] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.124018 28962 log.cc:824] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:20.124298 28961 tablet_bootstrap.cc:492] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: No bootstrap required, opened a new log
I20260924 21:17:20.124372 28961 ts_tablet_manager.cc:1397] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:20.124359 28963 tablet_bootstrap.cc:492] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: Bootstrap starting.
I20260924 21:17:20.124665 28962 tablet_bootstrap.cc:492] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: No bootstrap required, opened a new log
I20260924 21:17:20.124747 28962 ts_tablet_manager.cc:1397] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:20.124979 28963 tablet_bootstrap.cc:654] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.125284 28963 log.cc:824] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:20.126575 28963 tablet_bootstrap.cc:492] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: No bootstrap required, opened a new log
I20260924 21:17:20.126475 28961 raft_consensus.cc:348] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } }
I20260924 21:17:20.126641 28963 ts_tablet_manager.cc:1397] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:20.126634 28961 raft_consensus.cc:374] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.126673 28961 raft_consensus.cc:729] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ddadc9e6fb4233bfe2c9a0a52c97b2, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.126668 28962 raft_consensus.cc:348] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } }
I20260924 21:17:20.126821 28961 consensus_queue.cc:260] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } }
I20260924 21:17:20.126859 28962 raft_consensus.cc:374] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.126901 28962 raft_consensus.cc:729] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09cbcdb8a4764fcaa54c61e797ff6d40, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.127017 28962 consensus_queue.cc:260] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } }
I20260924 21:17:20.127086 28961 ts_tablet_manager.cc:1428] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:17:20.127246 28933 heartbeater.cc:503] Master 127.20.115.254:41495 was elected leader, sending a full tablet report...
I20260924 21:17:20.127281 28962 ts_tablet_manager.cc:1428] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:17:20.127493 28653 heartbeater.cc:503] Master 127.20.115.254:41495 was elected leader, sending a full tablet report...
I20260924 21:17:20.128736 28963 raft_consensus.cc:348] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } }
I20260924 21:17:20.128912 28963 raft_consensus.cc:374] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.128962 28963 raft_consensus.cc:729] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37f18682b274a818393b757f49f4a51, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.129073 28963 consensus_queue.cc:260] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } }
I20260924 21:17:20.129335 28793 heartbeater.cc:503] Master 127.20.115.254:41495 was elected leader, sending a full tablet report...
I20260924 21:17:20.129573 28963 ts_tablet_manager.cc:1428] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260924 21:17:20.174448 28794 tablet.cc:2369] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:17:20.223649 28968 raft_consensus.cc:482] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:20.223779 28968 raft_consensus.cc:504] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } }
I20260924 21:17:20.224162 28968 leader_election.cc:288] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831), c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623)
I20260924 21:17:20.228229 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.224466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:47107 (local address 127.20.115.194:40623)
0924 21:17:20.224518 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.224520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.224532 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:17:20.224569 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.224570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.224577 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.224658 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:20.224874 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.225623 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.227242 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.227824 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.227898 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.227910 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.228110 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.228156 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:17:20.228237 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.224447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:60171 (local address 127.20.115.195:38831)
0924 21:17:20.224497 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.224500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.224960 (+ 460us) server_negotiation.cc:407] Connection header received
0924 21:17:20.225018 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.225019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.225030 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.225087 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:20.225268 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.225942 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.227241 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.227792 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.227927 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.227936 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.228123 (+ 187us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.228160 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:17:20.228255 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.224386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40623 (local address 127.20.115.193:47107)
0924 21:17:20.224436 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.224450 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.224510 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.224679 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.224680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.224682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:20.224757 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.224759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.225837 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.225838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.227131 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.227134 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.227905 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.227906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.228020 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.228025 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.228089 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.228131 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":5}
I20260924 21:17:20.228251 28970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.224384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:38831 (local address 127.20.115.193:60171)
0924 21:17:20.224866 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.224879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.224970 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.225130 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.225130 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.225132 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:20.225200 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.225202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.226039 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.226039 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.227158 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.227160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.227905 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.227906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.228020 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.228026 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.228089 (+ 63us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.228131 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":182,"thread_start_us":118,"threads_started":1}
I20260924 21:17:20.228672 28747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d7b7e78f8f46ea82914827f1cb56e5" candidate_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37f18682b274a818393b757f49f4a51" is_pre_election: true
I20260924 21:17:20.228884 28887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d7b7e78f8f46ea82914827f1cb56e5" candidate_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" is_pre_election: true
I20260924 21:17:20.228922 28747 raft_consensus.cc:2454] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09cbcdb8a4764fcaa54c61e797ff6d40 in term 0.
I20260924 21:17:20.229046 28887 raft_consensus.cc:2454] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09cbcdb8a4764fcaa54c61e797ff6d40 in term 0.
I20260924 21:17:20.229198 28539 leader_election.cc:302] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09cbcdb8a4764fcaa54c61e797ff6d40, c37f18682b274a818393b757f49f4a51; no voters:
I20260924 21:17:20.229389 28968 raft_consensus.cc:2781] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:20.229431 28968 raft_consensus.cc:482] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:20.229447 28968 raft_consensus.cc:3037] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.230373 28968 raft_consensus.cc:504] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } }
I20260924 21:17:20.230526 28968 leader_election.cc:288] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [CANDIDATE]: Term 1 election: Requested vote from peers c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831), c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623)
I20260924 21:17:20.230669 28887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d7b7e78f8f46ea82914827f1cb56e5" candidate_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2"
I20260924 21:17:20.230697 28747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d7b7e78f8f46ea82914827f1cb56e5" candidate_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37f18682b274a818393b757f49f4a51"
I20260924 21:17:20.230759 28887 raft_consensus.cc:3037] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.230772 28747 raft_consensus.cc:3037] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.231444 28747 raft_consensus.cc:2454] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09cbcdb8a4764fcaa54c61e797ff6d40 in term 1.
I20260924 21:17:20.231464 28887 raft_consensus.cc:2454] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09cbcdb8a4764fcaa54c61e797ff6d40 in term 1.
I20260924 21:17:20.231612 28539 leader_election.cc:302] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09cbcdb8a4764fcaa54c61e797ff6d40, c37f18682b274a818393b757f49f4a51; no voters:
I20260924 21:17:20.231748 28968 raft_consensus.cc:2781] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:20.231900 28968 raft_consensus.cc:686] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 LEADER]: Becoming Leader. State: Replica: 09cbcdb8a4764fcaa54c61e797ff6d40, State: Running, Role: LEADER
I20260924 21:17:20.231974 28968 consensus_queue.cc:237] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } }
I20260924 21:17:20.232885 28450 catalog_manager.cc:5495] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } health_report { overall_health: HEALTHY } } }
I20260924 21:17:20.267096 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.264583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58364 (local address 127.20.115.193:36901)
0924 21:17:20.264651 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.264653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.264664 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:20.264750 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.264751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.264769 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.264933 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:20.265078 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.265551 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.266316 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.266513 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.266589 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.266764 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:20.266867 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:20.266957 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.266982 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.267002 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:17:20.268893 28887 raft_consensus.cc:1261] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Refusing update from remote peer 09cbcdb8a4764fcaa54c61e797ff6d40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:20.268918 28747 raft_consensus.cc:1261] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Refusing update from remote peer 09cbcdb8a4764fcaa54c61e797ff6d40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:20.269306 28971 consensus_queue.cc:1035] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 21:17:20.269410 28654 tablet.cc:2369] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:17:20.269486 28968 consensus_queue.cc:1035] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:20.272326 28976 mvcc.cc:205] Tried to move back new op lower bound from 7333005886538579968 to 7333005886390849536. Current Snapshot: MvccSnapshot[applied={T|T < 7333005886538579968}]
I20260924 21:17:20.272614 28975 mvcc.cc:205] Tried to move back new op lower bound from 7333005886538579968 to 7333005886390849536. Current Snapshot: MvccSnapshot[applied={T|T < 7333005886538579968 or (T in {7333005886538579968})}]
I20260924 21:17:20.274655 28450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:20.274765 28450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:20.275563 28450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a: Sending DeleteTablet for 3 replicas of tablet f4d7b7e78f8f46ea82914827f1cb56e5
I20260924 21:17:20.275888 28727 tablet_service.cc:1514] Processing DeleteTablet for tablet f4d7b7e78f8f46ea82914827f1cb56e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:48882
I20260924 21:17:20.275876 28867 tablet_service.cc:1514] Processing DeleteTablet for tablet f4d7b7e78f8f46ea82914827f1cb56e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:50688
I20260924 21:17:20.276224 28587 tablet_service.cc:1514] Processing DeleteTablet for tablet f4d7b7e78f8f46ea82914827f1cb56e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:58350
I20260924 21:17:20.276449 28985 tablet_replica.cc:330] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: stopping tablet replica
I20260924 21:17:20.276446 28986 tablet_replica.cc:330] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: stopping tablet replica
I20260924 21:17:20.276559 28985 raft_consensus.cc:2229] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:20.276645 28986 raft_consensus.cc:2229] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:17:20.276685 28985 raft_consensus.cc:2258] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:20.276643 28450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39282:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:17:20.276809 28450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:20.276859 28984 tablet_replica.cc:330] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: stopping tablet replica
I20260924 21:17:20.276947 28984 raft_consensus.cc:2229] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:20.277110 28986 raft_consensus.cc:2258] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:20.277542 28986 ts_tablet_manager.cc:1905] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:20.277674 28985 ts_tablet_manager.cc:1905] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:20.278435 28727 tablet_service.cc:1467] Processing CreateTablet for tablet 2eac6c1f6b884415be6bedaaeae59efb (DEFAULT_TABLE table=test-table [id=162082a9ebd74d5b8061d417e1247739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.278556 28727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eac6c1f6b884415be6bedaaeae59efb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.279419 28984 raft_consensus.cc:2258] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:20.279682 28985 ts_tablet_manager.cc:1918] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:20.279805 28985 log.cc:1196] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/f4d7b7e78f8f46ea82914827f1cb56e5
I20260924 21:17:20.279892 28963 tablet_bootstrap.cc:492] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: Bootstrap starting.
I20260924 21:17:20.279923 28984 ts_tablet_manager.cc:1905] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:20.280226 28985 ts_tablet_manager.cc:1939] T f4d7b7e78f8f46ea82914827f1cb56e5 P c37f18682b274a818393b757f49f4a51: Deleting consensus metadata
I20260924 21:17:20.280299 28963 tablet_bootstrap.cc:654] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.280599 28986 ts_tablet_manager.cc:1918] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:20.280675 28986 log.cc:1196] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/f4d7b7e78f8f46ea82914827f1cb56e5
I20260924 21:17:20.280885 28435 catalog_manager.cc:4841] TS c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623): tablet f4d7b7e78f8f46ea82914827f1cb56e5 (table test-table [id=a4449213502646c8bc6640f7c88883c5]) successfully deleted
I20260924 21:17:20.280969 28986 ts_tablet_manager.cc:1939] T f4d7b7e78f8f46ea82914827f1cb56e5 P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting consensus metadata
I20260924 21:17:20.280985 28867 tablet_service.cc:1467] Processing CreateTablet for tablet 2eac6c1f6b884415be6bedaaeae59efb (DEFAULT_TABLE table=test-table [id=162082a9ebd74d5b8061d417e1247739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.281111 28867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eac6c1f6b884415be6bedaaeae59efb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.281785 28984 ts_tablet_manager.cc:1918] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:20.281860 28984 log.cc:1196] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/f4d7b7e78f8f46ea82914827f1cb56e5
I20260924 21:17:20.282105 28984 ts_tablet_manager.cc:1939] T f4d7b7e78f8f46ea82914827f1cb56e5 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting consensus metadata
I20260924 21:17:20.282388 28961 tablet_bootstrap.cc:492] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Bootstrap starting.
I20260924 21:17:20.282518 28437 catalog_manager.cc:4841] TS 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901): tablet f4d7b7e78f8f46ea82914827f1cb56e5 (table test-table [id=a4449213502646c8bc6640f7c88883c5]) successfully deleted
I20260924 21:17:20.282622 28587 tablet_service.cc:1467] Processing CreateTablet for tablet 2eac6c1f6b884415be6bedaaeae59efb (DEFAULT_TABLE table=test-table [id=162082a9ebd74d5b8061d417e1247739]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.282719 28587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2eac6c1f6b884415be6bedaaeae59efb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.282815 28961 tablet_bootstrap.cc:654] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.283483 28961 tablet_bootstrap.cc:492] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: No bootstrap required, opened a new log
I20260924 21:17:20.283540 28961 ts_tablet_manager.cc:1397] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:20.283685 28961 raft_consensus.cc:348] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.283756 28961 raft_consensus.cc:374] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.283778 28961 raft_consensus.cc:729] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ddadc9e6fb4233bfe2c9a0a52c97b2, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.283766 28962 tablet_bootstrap.cc:492] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: Bootstrap starting.
I20260924 21:17:20.283820 28961 consensus_queue.cc:260] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.283902 28961 ts_tablet_manager.cc:1428] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:20.284168 28962 tablet_bootstrap.cc:654] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.284498 28963 tablet_bootstrap.cc:492] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: No bootstrap required, opened a new log
I20260924 21:17:20.284545 28963 ts_tablet_manager.cc:1397] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260924 21:17:20.284698 28963 raft_consensus.cc:348] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.284803 28963 raft_consensus.cc:374] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.284811 28962 tablet_bootstrap.cc:492] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: No bootstrap required, opened a new log
I20260924 21:17:20.284832 28963 raft_consensus.cc:729] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37f18682b274a818393b757f49f4a51, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.284852 28962 ts_tablet_manager.cc:1397] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:20.284878 28963 consensus_queue.cc:260] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.284986 28963 ts_tablet_manager.cc:1428] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:20.284991 28962 raft_consensus.cc:348] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.285061 28962 raft_consensus.cc:374] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.285087 28962 raft_consensus.cc:729] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09cbcdb8a4764fcaa54c61e797ff6d40, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.285128 28962 consensus_queue.cc:260] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.285218 28962 ts_tablet_manager.cc:1428] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.285306 28436 catalog_manager.cc:4841] TS c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831): tablet f4d7b7e78f8f46ea82914827f1cb56e5 (table test-table [id=a4449213502646c8bc6640f7c88883c5]) successfully deleted
W20260924 21:17:20.320479 28934 tablet.cc:2369] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:17:20.340124 28969 raft_consensus.cc:482] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:20.340245 28969 raft_consensus.cc:504] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.340596 28969 leader_election.cc:288] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901), c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831)
I20260924 21:17:20.344475 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.341004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:51047 (local address 127.20.115.193:36901)
0924 21:17:20.341050 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.341052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.341061 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:20.341086 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.341087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.341096 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.341162 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:20.341355 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.342090 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.343564 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.344107 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.344190 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.344199 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.344366 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.344406 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:17:20.344492 28990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.340826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:36901 (local address 127.20.115.194:51047)
0924 21:17:20.340974 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.340984 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.341026 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.341166 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.341167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.341169 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:20.341264 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.341265 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.342187 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.342188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.343476 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.343478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.344218 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.344219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.344289 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.344293 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.344348 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.344388 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":63,"threads_started":1}
I20260924 21:17:20.344893 28607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eac6c1f6b884415be6bedaaeae59efb" candidate_uuid: "c37f18682b274a818393b757f49f4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" is_pre_election: true
I20260924 21:17:20.345031 28607 raft_consensus.cc:2454] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37f18682b274a818393b757f49f4a51 in term 0.
I20260924 21:17:20.345242 28681 leader_election.cc:302] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09cbcdb8a4764fcaa54c61e797ff6d40, c37f18682b274a818393b757f49f4a51; no voters:
I20260924 21:17:20.345315 28686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.340791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:38831 (local address 127.20.115.194:47071)
0924 21:17:20.340842 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.340865 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.340915 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.341216 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.341216 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.341217 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:20.341265 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.341266 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.343105 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.343106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.343934 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.343936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.344664 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.344665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.344747 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.344750 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.345207 (+ 457us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.345256 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260924 21:17:20.345351 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.340918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:47071 (local address 127.20.115.195:38831)
0924 21:17:20.340975 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.340977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.340986 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:20.341016 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.341017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.341026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.341083 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:20.342189 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.342992 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.344022 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.344557 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.344622 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.344633 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.345242 (+ 609us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.345289 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:17:20.345366 28969 raft_consensus.cc:2781] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:20.345399 28969 raft_consensus.cc:482] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:20.345422 28969 raft_consensus.cc:3037] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.345579 28887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eac6c1f6b884415be6bedaaeae59efb" candidate_uuid: "c37f18682b274a818393b757f49f4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" is_pre_election: true
I20260924 21:17:20.345670 28887 raft_consensus.cc:2454] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37f18682b274a818393b757f49f4a51 in term 0.
I20260924 21:17:20.346280 28969 raft_consensus.cc:504] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.346426 28969 leader_election.cc:288] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [CANDIDATE]: Term 1 election: Requested vote from peers 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901), c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831)
I20260924 21:17:20.346549 28607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eac6c1f6b884415be6bedaaeae59efb" candidate_uuid: "c37f18682b274a818393b757f49f4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40"
I20260924 21:17:20.346621 28607 raft_consensus.cc:3037] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.346603 28887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2eac6c1f6b884415be6bedaaeae59efb" candidate_uuid: "c37f18682b274a818393b757f49f4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2"
I20260924 21:17:20.346674 28887 raft_consensus.cc:3037] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.347203 28607 raft_consensus.cc:2454] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37f18682b274a818393b757f49f4a51 in term 1.
I20260924 21:17:20.347386 28681 leader_election.cc:302] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09cbcdb8a4764fcaa54c61e797ff6d40, c37f18682b274a818393b757f49f4a51; no voters:
I20260924 21:17:20.347471 28887 raft_consensus.cc:2454] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37f18682b274a818393b757f49f4a51 in term 1.
I20260924 21:17:20.347482 28969 raft_consensus.cc:2781] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:20.347651 28969 raft_consensus.cc:686] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 1 LEADER]: Becoming Leader. State: Replica: c37f18682b274a818393b757f49f4a51, State: Running, Role: LEADER
I20260924 21:17:20.347729 28969 consensus_queue.cc:237] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.348541 28450 catalog_manager.cc:5495] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to c37f18682b274a818393b757f49f4a51 (127.20.115.194). New cstate: current_term: 1 leader_uuid: "c37f18682b274a818393b757f49f4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } health_report { overall_health: UNKNOWN } } }
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284639, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.254@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:20.388613 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.381698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39288 (local address 127.20.115.254:41495)
0924 21:17:20.381768 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.381771 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.381901 (+ 130us) server_negotiation.cc:407] Connection header received
0924 21:17:20.381991 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.381992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.382002 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.382052 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:20.382520 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.383015 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.383704 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.383899 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.383984 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.386006 (+ 2022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:20.386007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:20.386007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:20.386014 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:20.387379 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.387764 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.387765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.387766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.387821 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.388042 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.388043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.388043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.388153 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:20.388230 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.388405 (+ 175us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.388506 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:17:20.388693 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.381614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:41495 (local address 127.0.0.1:39288)
0924 21:17:20.381783 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.381794 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.381911 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.382056 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.382057 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.382365 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:20.382410 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.382412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.383100 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.383101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.383622 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.383623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.383985 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.383985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.384039 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.384577 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:20.384582 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:20.385917 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:20.387479 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:20.387486 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:20.387488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:20.387674 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:20.387916 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:20.387917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:20.387918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:20.387978 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:20.388220 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:20.388227 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:20.388274 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.388379 (+ 105us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.388563 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":82,"thread_start_us":101,"threads_started":1}
I20260924 21:17:20.390756 28450 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:39288:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260924 21:17:20.390982 28450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:20.392446 28727 tablet_service.cc:1467] Processing CreateTablet for tablet 42199ac5eda249d987f08ef4837d26e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=da733294b7164acea2bc83b7ed834a05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:17:20.392468 28867 tablet_service.cc:1467] Processing CreateTablet for tablet 42199ac5eda249d987f08ef4837d26e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=da733294b7164acea2bc83b7ed834a05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:17:20.392477 28587 tablet_service.cc:1467] Processing CreateTablet for tablet 42199ac5eda249d987f08ef4837d26e3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=da733294b7164acea2bc83b7ed834a05]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260924 21:17:20.392607 28867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42199ac5eda249d987f08ef4837d26e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.392607 28727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42199ac5eda249d987f08ef4837d26e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.392629 28587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 42199ac5eda249d987f08ef4837d26e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.394023 28962 tablet_bootstrap.cc:492] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: Bootstrap starting.
I20260924 21:17:20.394147 28963 tablet_bootstrap.cc:492] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: Bootstrap starting.
I20260924 21:17:20.394346 28962 tablet_bootstrap.cc:654] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.394598 28963 tablet_bootstrap.cc:654] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.394608 28961 tablet_bootstrap.cc:492] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Bootstrap starting.
I20260924 21:17:20.395021 28961 tablet_bootstrap.cc:654] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.395485 28963 tablet_bootstrap.cc:492] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: No bootstrap required, opened a new log
I20260924 21:17:20.395531 28963 ts_tablet_manager.cc:1397] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:20.395634 28962 tablet_bootstrap.cc:492] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: No bootstrap required, opened a new log
I20260924 21:17:20.395675 28962 ts_tablet_manager.cc:1397] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:20.395658 28963 raft_consensus.cc:348] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.395720 28963 raft_consensus.cc:374] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.395735 28963 raft_consensus.cc:729] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37f18682b274a818393b757f49f4a51, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.395769 28962 raft_consensus.cc:348] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.395794 28963 consensus_queue.cc:260] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.395828 28962 raft_consensus.cc:374] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.395846 28962 raft_consensus.cc:729] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09cbcdb8a4764fcaa54c61e797ff6d40, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.395884 28963 ts_tablet_manager.cc:1428] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.395886 28962 consensus_queue.cc:260] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.395920 28969 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.395964 28962 ts_tablet_manager.cc:1428] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.395978 28969 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:17:20.396054 28980 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.396107 28980 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:17:20.396859 28961 tablet_bootstrap.cc:492] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: No bootstrap required, opened a new log
I20260924 21:17:20.396915 28961 ts_tablet_manager.cc:1397] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:17:20.397053 28961 raft_consensus.cc:348] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.397118 28961 raft_consensus.cc:374] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.397137 28961 raft_consensus.cc:729] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ddadc9e6fb4233bfe2c9a0a52c97b2, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.397182 28961 consensus_queue.cc:260] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.397289 28961 ts_tablet_manager.cc:1428] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.397436 28967 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.397522 28967 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:17:20.550537 28969 raft_consensus.cc:482] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:20.550637 28969 raft_consensus.cc:504] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.550804 28969 leader_election.cc:288] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901), c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831)
I20260924 21:17:20.551080 28607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42199ac5eda249d987f08ef4837d26e3" candidate_uuid: "c37f18682b274a818393b757f49f4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" is_pre_election: true
I20260924 21:17:20.551079 28887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42199ac5eda249d987f08ef4837d26e3" candidate_uuid: "c37f18682b274a818393b757f49f4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" is_pre_election: true
I20260924 21:17:20.551170 28607 raft_consensus.cc:2454] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37f18682b274a818393b757f49f4a51 in term 0.
I20260924 21:17:20.551201 28887 raft_consensus.cc:2454] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c37f18682b274a818393b757f49f4a51 in term 0.
I20260924 21:17:20.551332 28681 leader_election.cc:302] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09cbcdb8a4764fcaa54c61e797ff6d40, c37f18682b274a818393b757f49f4a51; no voters:
I20260924 21:17:20.551446 28969 raft_consensus.cc:2781] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:20.551535 28969 raft_consensus.cc:482] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:20.551566 28969 raft_consensus.cc:3037] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.552253 28969 raft_consensus.cc:504] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.552399 28969 leader_election.cc:288] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [CANDIDATE]: Term 1 election: Requested vote from peers 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901), c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831)
I20260924 21:17:20.552537 28887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42199ac5eda249d987f08ef4837d26e3" candidate_uuid: "c37f18682b274a818393b757f49f4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2"
I20260924 21:17:20.552557 28607 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "42199ac5eda249d987f08ef4837d26e3" candidate_uuid: "c37f18682b274a818393b757f49f4a51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40"
I20260924 21:17:20.552623 28607 raft_consensus.cc:3037] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.552634 28887 raft_consensus.cc:3037] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.553277 28887 raft_consensus.cc:2454] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37f18682b274a818393b757f49f4a51 in term 1.
I20260924 21:17:20.553333 28607 raft_consensus.cc:2454] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c37f18682b274a818393b757f49f4a51 in term 1.
I20260924 21:17:20.553483 28680 leader_election.cc:302] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c37f18682b274a818393b757f49f4a51, c7ddadc9e6fb4233bfe2c9a0a52c97b2; no voters:
I20260924 21:17:20.553592 28969 raft_consensus.cc:2781] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:20.553685 28969 raft_consensus.cc:686] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [term 1 LEADER]: Becoming Leader. State: Replica: c37f18682b274a818393b757f49f4a51, State: Running, Role: LEADER
I20260924 21:17:20.553735 28969 consensus_queue.cc:237] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.554011 28991 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: TxnStatusTablet state changed. Reason: New leader c37f18682b274a818393b757f49f4a51. Latest consensus state: current_term: 1 leader_uuid: "c37f18682b274a818393b757f49f4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.554085 28991 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:17:20.554222 29003 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260924 21:17:20.554342 29003 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260924 21:17:20.554311 28449 catalog_manager.cc:5495] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 reported cstate change: term changed from 0 to 1, leader changed from <none> to c37f18682b274a818393b757f49f4a51 (127.20.115.194). New cstate: current_term: 1 leader_uuid: "c37f18682b274a818393b757f49f4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } health_report { overall_health: UNKNOWN } } }
I20260924 21:17:20.578804 28998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.576061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40623 (local address 127.0.0.1:48884)
0924 21:17:20.576137 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.576150 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.576201 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.576376 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.576377 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.576378 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:17:20.576423 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.576424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.577328 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.577329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.577883 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.577884 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.578338 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.578339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.578393 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.578405 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:17:20.578676 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:17:20.578682 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.578717 (+ 35us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.578753 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:17:20.578840 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.576152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48884 (local address 127.20.115.194:40623)
0924 21:17:20.576205 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.576207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.576217 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:20.576279 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.576280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.576301 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.576376 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:20.576463 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.577259 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.577963 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.578265 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.578334 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.578493 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:20.578613 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:20.578686 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.578724 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.578745 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:17:20.579849 28887 raft_consensus.cc:1261] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Refusing update from remote peer c37f18682b274a818393b757f49f4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:20.579989 28607 raft_consensus.cc:1261] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Refusing update from remote peer c37f18682b274a818393b757f49f4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:20.580094 28991 consensus_queue.cc:1035] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:20.580245 28969 consensus_queue.cc:1035] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:20.580650 28967 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: TxnStatusTablet state changed. Reason: New leader c37f18682b274a818393b757f49f4a51. Latest consensus state: current_term: 1 leader_uuid: "c37f18682b274a818393b757f49f4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.580762 28967 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:17:20.580912 28980 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: TxnStatusTablet state changed. Reason: New leader c37f18682b274a818393b757f49f4a51. Latest consensus state: current_term: 1 leader_uuid: "c37f18682b274a818393b757f49f4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.581007 28980 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:17:20.581451 28969 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c37f18682b274a818393b757f49f4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.581542 28969 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:17:20.582072 28967 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c37f18682b274a818393b757f49f4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.582172 28967 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:17:20.583220 28991 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c37f18682b274a818393b757f49f4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.583297 28991 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P c37f18682b274a818393b757f49f4a51: This TxnStatusTablet replica's current role is: LEADER
I20260924 21:17:20.583642 28974 mvcc.cc:205] Tried to move back new op lower bound from 7333005887813337088 to 7333005887708483584. Current Snapshot: MvccSnapshot[applied={T|T < 7333005887813337088}]
I20260924 21:17:20.583736 28980 tablet_replica.cc:439] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c37f18682b274a818393b757f49f4a51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } } }
I20260924 21:17:20.583804 28980 tablet_replica.cc:442] T 42199ac5eda249d987f08ef4837d26e3 P 09cbcdb8a4764fcaa54c61e797ff6d40: This TxnStatusTablet replica's current role is: FOLLOWER
I20260924 21:17:20.585475 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.583021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48894 (local address 127.20.115.194:40623)
0924 21:17:20.583054 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.583055 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.583063 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:20.583084 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.583084 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.583095 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.583141 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:20.583303 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.583988 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.584737 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.585017 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.585094 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.585247 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:20.585280 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:20.585326 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.585383 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.585401 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:17:20.590667 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.588067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:33513 (local address 127.20.115.194:40623)
0924 21:17:20.588119 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.588121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.588231 (+ 110us) server_negotiation.cc:407] Connection header received
0924 21:17:20.588259 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.588261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.588271 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.588332 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:20.588451 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.589154 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.589915 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.590203 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.590274 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.590438 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:20.590468 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:20.590521 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.590569 (+ 48us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.590585 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:17:20.590677 29011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.587992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40623 (local address 127.20.115.194:33513)
0924 21:17:20.588171 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.588180 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.588226 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.588340 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.588340 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.588341 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:17:20.588391 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.588392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.589248 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.589249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.589832 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.589833 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.590299 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.590300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.590346 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.590358 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:17:20.590530 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:17:20.590531 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.590565 (+ 34us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.590597 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
I20260924 21:17:20.592886 28607 raft_consensus.cc:1261] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Refusing update from remote peer c37f18682b274a818393b757f49f4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:20.592887 28887 raft_consensus.cc:1261] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Refusing update from remote peer c37f18682b274a818393b757f49f4a51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:20.593078 28991 consensus_queue.cc:1035] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:20.593166 28969 consensus_queue.cc:1035] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:20.604094 28450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:20.604195 28450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39282:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:20.604802 28450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a: Sending DeleteTablet for 3 replicas of tablet 2eac6c1f6b884415be6bedaaeae59efb
I20260924 21:17:20.605047 28867 tablet_service.cc:1514] Processing DeleteTablet for tablet 2eac6c1f6b884415be6bedaaeae59efb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:50688
I20260924 21:17:20.605130 28587 tablet_service.cc:1514] Processing DeleteTablet for tablet 2eac6c1f6b884415be6bedaaeae59efb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:58350
I20260924 21:17:20.605183 28984 tablet_replica.cc:330] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: stopping tablet replica
I20260924 21:17:20.605232 28986 tablet_replica.cc:330] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: stopping tablet replica
I20260924 21:17:20.605243 28984 raft_consensus.cc:2229] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:20.605288 28986 raft_consensus.cc:2229] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:20.605304 28984 raft_consensus.cc:2258] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:20.605325 28986 raft_consensus.cc:2258] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:20.605623 28984 ts_tablet_manager.cc:1905] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:20.605715 28986 ts_tablet_manager.cc:1905] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:20.607203 28986 ts_tablet_manager.cc:1918] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:17:20.607266 28986 log.cc:1196] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/2eac6c1f6b884415be6bedaaeae59efb
I20260924 21:17:20.607419 28984 ts_tablet_manager.cc:1918] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:17:20.607491 28984 log.cc:1196] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/2eac6c1f6b884415be6bedaaeae59efb
I20260924 21:17:20.607513 28986 ts_tablet_manager.cc:1939] T 2eac6c1f6b884415be6bedaaeae59efb P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting consensus metadata
I20260924 21:17:20.607721 28984 ts_tablet_manager.cc:1939] T 2eac6c1f6b884415be6bedaaeae59efb P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting consensus metadata
I20260924 21:17:20.607831 28437 catalog_manager.cc:4841] TS 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901): tablet 2eac6c1f6b884415be6bedaaeae59efb (table test-table [id=162082a9ebd74d5b8061d417e1247739]) successfully deleted
I20260924 21:17:20.608044 28436 catalog_manager.cc:4841] TS c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831): tablet 2eac6c1f6b884415be6bedaaeae59efb (table test-table [id=162082a9ebd74d5b8061d417e1247739]) successfully deleted
I20260924 21:17:20.608431 28726 tablet_service.cc:1514] Processing DeleteTablet for tablet 2eac6c1f6b884415be6bedaaeae59efb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:48882
I20260924 21:17:20.608539 28985 tablet_replica.cc:330] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: stopping tablet replica
I20260924 21:17:20.608624 28985 raft_consensus.cc:2229] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:17:20.608721 28985 raft_consensus.cc:2258] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:20.609014 28985 ts_tablet_manager.cc:1905] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:20.610219 28985 ts_tablet_manager.cc:1918] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:17:20.610280 28985 log.cc:1196] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/2eac6c1f6b884415be6bedaaeae59efb
I20260924 21:17:20.610494 28985 ts_tablet_manager.cc:1939] T 2eac6c1f6b884415be6bedaaeae59efb P c37f18682b274a818393b757f49f4a51: Deleting consensus metadata
I20260924 21:17:20.610795 28435 catalog_manager.cc:4841] TS c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623): tablet 2eac6c1f6b884415be6bedaaeae59efb (table test-table [id=162082a9ebd74d5b8061d417e1247739]) successfully deleted
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 21:17:20.629703 28500 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:20.623484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39292 (local address 127.20.115.254:41495)
0924 21:17:20.623566 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.623568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.623659 (+ 91us) server_negotiation.cc:407] Connection header received
0924 21:17:20.623729 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.623729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.623738 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.623785 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:20.624472 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.625015 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.626090 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.626264 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.626308 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.629639 (+ 3331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39292: BlockingRecv error: recv got EOF from 127.0.0.1:39292 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 24 21:17:20 dist-test-slave-cxpd krb5kdc[28405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM
I20260924 21:17:20.637663 28500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.630495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39308 (local address 127.20.115.254:41495)
0924 21:17:20.630550 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.630552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.630665 (+ 113us) server_negotiation.cc:407] Connection header received
0924 21:17:20.630695 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.630696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.630707 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.630776 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:20.631195 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.631934 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.632597 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.632907 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.632985 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.635051 (+ 2066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:20.635052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:20.635053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:20.635060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:20.636524 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.636881 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.636882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.636883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.636940 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:20.637143 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:20.637144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:20.637145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:20.637253 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:20.637358 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.637463 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.637543 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:17:20.639127 28450 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:17:20.639307 28450 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:20.640606 28587 tablet_service.cc:1467] Processing CreateTablet for tablet 3eaba7e2dee142ef8b9c81554bf276b8 (DEFAULT_TABLE table=test-table [id=34507bf965354c70a226465510788aeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.640713 28867 tablet_service.cc:1467] Processing CreateTablet for tablet 3eaba7e2dee142ef8b9c81554bf276b8 (DEFAULT_TABLE table=test-table [id=34507bf965354c70a226465510788aeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.640712 28726 tablet_service.cc:1467] Processing CreateTablet for tablet 3eaba7e2dee142ef8b9c81554bf276b8 (DEFAULT_TABLE table=test-table [id=34507bf965354c70a226465510788aeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.640746 28587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eaba7e2dee142ef8b9c81554bf276b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.640866 28867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eaba7e2dee142ef8b9c81554bf276b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.640866 28726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3eaba7e2dee142ef8b9c81554bf276b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.642221 28962 tablet_bootstrap.cc:492] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: Bootstrap starting.
I20260924 21:17:20.642231 28961 tablet_bootstrap.cc:492] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Bootstrap starting.
I20260924 21:17:20.642246 28963 tablet_bootstrap.cc:492] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: Bootstrap starting.
I20260924 21:17:20.642556 28962 tablet_bootstrap.cc:654] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.642627 28961 tablet_bootstrap.cc:654] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.642627 28963 tablet_bootstrap.cc:654] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.643272 28962 tablet_bootstrap.cc:492] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: No bootstrap required, opened a new log
I20260924 21:17:20.643326 28962 ts_tablet_manager.cc:1397] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:20.643455 28963 tablet_bootstrap.cc:492] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: No bootstrap required, opened a new log
I20260924 21:17:20.643437 28962 raft_consensus.cc:348] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.643498 28963 ts_tablet_manager.cc:1397] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 21:17:20.643510 28962 raft_consensus.cc:374] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.643533 28962 raft_consensus.cc:729] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09cbcdb8a4764fcaa54c61e797ff6d40, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.643585 28961 tablet_bootstrap.cc:492] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: No bootstrap required, opened a new log
I20260924 21:17:20.643575 28962 consensus_queue.cc:260] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.643621 28961 ts_tablet_manager.cc:1397] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:20.643667 28962 ts_tablet_manager.cc:1428] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.643637 28963 raft_consensus.cc:348] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.643698 28963 raft_consensus.cc:374] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.643721 28963 raft_consensus.cc:729] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37f18682b274a818393b757f49f4a51, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.643764 28963 consensus_queue.cc:260] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.643769 28961 raft_consensus.cc:348] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.643838 28961 raft_consensus.cc:374] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.643842 28963 ts_tablet_manager.cc:1428] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.643855 28961 raft_consensus.cc:729] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ddadc9e6fb4233bfe2c9a0a52c97b2, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.643891 28961 consensus_queue.cc:260] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.643962 28961 ts_tablet_manager.cc:1428] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.669498 28967 raft_consensus.cc:482] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:20.669612 28967 raft_consensus.cc:504] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.669957 28967 leader_election.cc:288] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901), c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623)
I20260924 21:17:20.673743 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.670249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:57925 (local address 127.20.115.193:36901)
0924 21:17:20.670298 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.670300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.670465 (+ 165us) server_negotiation.cc:407] Connection header received
0924 21:17:20.670497 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.670498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.670508 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.670584 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:20.670777 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.671390 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.672790 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.673350 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.673455 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.673466 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.673613 (+ 147us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.673656 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:17:20.673743 29031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.670132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:36901 (local address 127.20.115.195:57925)
0924 21:17:20.670360 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.670372 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.670442 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.670592 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.670592 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.670595 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:20.670679 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.670682 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.671482 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.671483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.672682 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.672685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.673476 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.673477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.673547 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.673551 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.673607 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.673645 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260924 21:17:20.674224 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.670287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:44231 (local address 127.20.115.194:40623)
0924 21:17:20.670333 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.670335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.670448 (+ 113us) server_negotiation.cc:407] Connection header received
0924 21:17:20.670568 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.670569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.670579 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.670627 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:20.671224 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.671969 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.673278 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.673845 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.673921 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.673929 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.674100 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.674139 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 21:17:20.674464 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.670134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40623 (local address 127.20.115.195:44231)
0924 21:17:20.670362 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:20.670372 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:20.670442 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:20.670657 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:20.670658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:20.670661 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:20.671103 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.671106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.672068 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.672069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.673197 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:20.673199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.673960 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.673961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:20.674025 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.674028 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:20.674343 (+ 315us) client_negotiation.cc:240] Negotiation successful
0924 21:17:20.674383 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":11,"spinlock_wait_cycles":896,"thread_start_us":82,"threads_started":1}
I20260924 21:17:20.674715 28747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eaba7e2dee142ef8b9c81554bf276b8" candidate_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37f18682b274a818393b757f49f4a51" is_pre_election: true
I20260924 21:17:20.674858 28747 raft_consensus.cc:2454] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ddadc9e6fb4233bfe2c9a0a52c97b2 in term 0.
I20260924 21:17:20.675030 28819 leader_election.cc:302] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c37f18682b274a818393b757f49f4a51, c7ddadc9e6fb4233bfe2c9a0a52c97b2; no voters:
I20260924 21:17:20.677012 28967 raft_consensus.cc:2781] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:20.676993 28606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eaba7e2dee142ef8b9c81554bf276b8" candidate_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" is_pre_election: true
I20260924 21:17:20.677078 28967 raft_consensus.cc:482] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:20.677088 28606 raft_consensus.cc:2454] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7ddadc9e6fb4233bfe2c9a0a52c97b2 in term 0.
I20260924 21:17:20.677106 28967 raft_consensus.cc:3037] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.678413 28967 raft_consensus.cc:504] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.678555 28967 leader_election.cc:288] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [CANDIDATE]: Term 1 election: Requested vote from peers 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901), c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623)
I20260924 21:17:20.678740 28606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eaba7e2dee142ef8b9c81554bf276b8" candidate_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40"
I20260924 21:17:20.678830 28606 raft_consensus.cc:3037] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.678861 28747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3eaba7e2dee142ef8b9c81554bf276b8" candidate_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37f18682b274a818393b757f49f4a51"
I20260924 21:17:20.678939 28747 raft_consensus.cc:3037] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.679708 28747 raft_consensus.cc:2454] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ddadc9e6fb4233bfe2c9a0a52c97b2 in term 1.
I20260924 21:17:20.679776 28606 raft_consensus.cc:2454] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7ddadc9e6fb4233bfe2c9a0a52c97b2 in term 1.
I20260924 21:17:20.679878 28819 leader_election.cc:302] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c37f18682b274a818393b757f49f4a51, c7ddadc9e6fb4233bfe2c9a0a52c97b2; no voters:
I20260924 21:17:20.679953 28967 raft_consensus.cc:2781] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:20.680051 28967 raft_consensus.cc:686] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 LEADER]: Becoming Leader. State: Replica: c7ddadc9e6fb4233bfe2c9a0a52c97b2, State: Running, Role: LEADER
I20260924 21:17:20.680110 28967 consensus_queue.cc:237] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.680840 28450 catalog_manager.cc:5495] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 reported cstate change: term changed from 0 to 1, leader changed from <none> to c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195). New cstate: current_term: 1 leader_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } health_report { overall_health: HEALTHY } } }
I20260924 21:17:20.694988 28958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.692513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50690 (local address 127.20.115.195:38831)
0924 21:17:20.692560 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.692562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.692571 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:20.692643 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.692644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.692667 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.692755 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:20.692917 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.693386 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.694167 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.694379 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.694472 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.694645 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:20.694747 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:20.694842 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.694882 (+ 40us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.694903 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:17:20.695940 28747 raft_consensus.cc:1261] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Refusing update from remote peer c7ddadc9e6fb4233bfe2c9a0a52c97b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:20.695947 28606 raft_consensus.cc:1261] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Refusing update from remote peer c7ddadc9e6fb4233bfe2c9a0a52c97b2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:20.696205 29032 consensus_queue.cc:1035] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:20.696324 28967 consensus_queue.cc:1035] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:20.699774 28450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:20.699859 28450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:20.700712 28450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a: Sending DeleteTablet for 3 replicas of tablet 3eaba7e2dee142ef8b9c81554bf276b8
I20260924 21:17:20.700958 28587 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eaba7e2dee142ef8b9c81554bf276b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:58350
I20260924 21:17:20.701048 28986 tablet_replica.cc:330] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: stopping tablet replica
I20260924 21:17:20.701102 28986 raft_consensus.cc:2229] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:20.701143 28986 raft_consensus.cc:2258] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:20.701340 28867 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eaba7e2dee142ef8b9c81554bf276b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:50688
I20260924 21:17:20.701426 28984 tablet_replica.cc:330] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: stopping tablet replica
I20260924 21:17:20.701471 28984 raft_consensus.cc:2229] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:17:20.701411 28448 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39308:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260924 21:17:20.701540 28984 raft_consensus.cc:2258] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260924 21:17:20.701565 28448 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:20.701932 28726 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eaba7e2dee142ef8b9c81554bf276b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:48882
I20260924 21:17:20.701972 28727 tablet_service.cc:1514] Processing DeleteTablet for tablet 3eaba7e2dee142ef8b9c81554bf276b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:20 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:48882
I20260924 21:17:20.702019 28985 tablet_replica.cc:330] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: stopping tablet replica
I20260924 21:17:20.702059 28985 raft_consensus.cc:2229] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:20.702101 28985 raft_consensus.cc:2258] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:20.703102 28985 ts_tablet_manager.cc:1905] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:20.703220 28587 tablet_service.cc:1467] Processing CreateTablet for tablet b80b002362bc451ab93f47a581fa8fe1 (DEFAULT_TABLE table=test-table [id=e2b4026faf814d788b48ee600c9c93ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.703325 28587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b80b002362bc451ab93f47a581fa8fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.704506 28962 tablet_bootstrap.cc:492] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40: Bootstrap starting.
I20260924 21:17:20.704710 28985 ts_tablet_manager.cc:1918] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:20.704770 28985 log.cc:1196] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/3eaba7e2dee142ef8b9c81554bf276b8
I20260924 21:17:20.704794 28962 tablet_bootstrap.cc:654] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.705027 28985 ts_tablet_manager.cc:1939] T 3eaba7e2dee142ef8b9c81554bf276b8 P c37f18682b274a818393b757f49f4a51: Deleting consensus metadata
I20260924 21:17:20.705205 28867 tablet_service.cc:1467] Processing CreateTablet for tablet b80b002362bc451ab93f47a581fa8fe1 (DEFAULT_TABLE table=test-table [id=e2b4026faf814d788b48ee600c9c93ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.705302 28867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b80b002362bc451ab93f47a581fa8fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.706436 28961 tablet_bootstrap.cc:492] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Bootstrap starting.
I20260924 21:17:20.706596 28435 catalog_manager.cc:4841] TS c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623): tablet 3eaba7e2dee142ef8b9c81554bf276b8 (table test-table [id=34507bf965354c70a226465510788aeb]) successfully deleted
W20260924 21:17:20.706707 28435 catalog_manager.cc:4808] TS c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623): delete failed for tablet 3eaba7e2dee142ef8b9c81554bf276b8 because the tablet was not found. No further retry: Not found: Tablet not found: 3eaba7e2dee142ef8b9c81554bf276b8
I20260924 21:17:20.706837 28961 tablet_bootstrap.cc:654] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.706848 28727 tablet_service.cc:1467] Processing CreateTablet for tablet b80b002362bc451ab93f47a581fa8fe1 (DEFAULT_TABLE table=test-table [id=e2b4026faf814d788b48ee600c9c93ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:20.706935 28727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b80b002362bc451ab93f47a581fa8fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:20.707551 28961 tablet_bootstrap.cc:492] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: No bootstrap required, opened a new log
I20260924 21:17:20.707599 28961 ts_tablet_manager.cc:1397] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:20.707731 28961 raft_consensus.cc:348] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.707791 28961 raft_consensus.cc:374] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.707811 28961 raft_consensus.cc:729] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7ddadc9e6fb4233bfe2c9a0a52c97b2, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.707851 28961 consensus_queue.cc:260] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.707930 28961 ts_tablet_manager.cc:1428] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.708060 28963 tablet_bootstrap.cc:492] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: Bootstrap starting.
I20260924 21:17:20.708380 28963 tablet_bootstrap.cc:654] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:20.709147 28963 tablet_bootstrap.cc:492] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: No bootstrap required, opened a new log
I20260924 21:17:20.709195 28963 ts_tablet_manager.cc:1397] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 21:17:20.709280 28963 raft_consensus.cc:348] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.709340 28963 raft_consensus.cc:374] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.709357 28963 raft_consensus.cc:729] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37f18682b274a818393b757f49f4a51, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.709391 28963 consensus_queue.cc:260] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.709473 28963 ts_tablet_manager.cc:1428] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.709757 28984 ts_tablet_manager.cc:1905] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:20.711119 28984 ts_tablet_manager.cc:1918] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:20.711172 28984 log.cc:1196] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/3eaba7e2dee142ef8b9c81554bf276b8
I20260924 21:17:20.711169 28986 ts_tablet_manager.cc:1905] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:20.711366 28962 tablet_bootstrap.cc:492] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40: No bootstrap required, opened a new log
I20260924 21:17:20.711386 28984 ts_tablet_manager.cc:1939] T 3eaba7e2dee142ef8b9c81554bf276b8 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting consensus metadata
I20260924 21:17:20.711412 28962 ts_tablet_manager.cc:1397] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.000s
I20260924 21:17:20.711526 28962 raft_consensus.cc:348] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.711587 28962 raft_consensus.cc:374] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:20.711603 28962 raft_consensus.cc:729] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09cbcdb8a4764fcaa54c61e797ff6d40, State: Initialized, Role: FOLLOWER
I20260924 21:17:20.711642 28962 consensus_queue.cc:260] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.711719 28962 ts_tablet_manager.cc:1428] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:20.711709 28436 catalog_manager.cc:4841] TS c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831): tablet 3eaba7e2dee142ef8b9c81554bf276b8 (table test-table [id=34507bf965354c70a226465510788aeb]) successfully deleted
I20260924 21:17:20.712599 28986 ts_tablet_manager.cc:1918] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:20.712649 28986 log.cc:1196] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/3eaba7e2dee142ef8b9c81554bf276b8
I20260924 21:17:20.712880 28986 ts_tablet_manager.cc:1939] T 3eaba7e2dee142ef8b9c81554bf276b8 P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting consensus metadata
I20260924 21:17:20.713399 28437 catalog_manager.cc:4841] TS 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901): tablet 3eaba7e2dee142ef8b9c81554bf276b8 (table test-table [id=34507bf965354c70a226465510788aeb]) successfully deleted
I20260924 21:17:20.967940 28980 raft_consensus.cc:482] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:20.968055 28980 raft_consensus.cc:504] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.968235 28980 leader_election.cc:288] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623), c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831)
I20260924 21:17:20.968509 28747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80b002362bc451ab93f47a581fa8fe1" candidate_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37f18682b274a818393b757f49f4a51" is_pre_election: true
I20260924 21:17:20.968511 28887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80b002362bc451ab93f47a581fa8fe1" candidate_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" is_pre_election: true
I20260924 21:17:20.968598 28747 raft_consensus.cc:2454] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09cbcdb8a4764fcaa54c61e797ff6d40 in term 0.
I20260924 21:17:20.968628 28887 raft_consensus.cc:2454] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09cbcdb8a4764fcaa54c61e797ff6d40 in term 0.
I20260924 21:17:20.968816 28540 leader_election.cc:302] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09cbcdb8a4764fcaa54c61e797ff6d40, c7ddadc9e6fb4233bfe2c9a0a52c97b2; no voters:
I20260924 21:17:20.968923 28980 raft_consensus.cc:2781] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:20.968977 28980 raft_consensus.cc:482] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:20.969003 28980 raft_consensus.cc:3037] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.969739 28980 raft_consensus.cc:504] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.969892 28980 leader_election.cc:288] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [CANDIDATE]: Term 1 election: Requested vote from peers c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623), c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831)
I20260924 21:17:20.970028 28747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80b002362bc451ab93f47a581fa8fe1" candidate_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37f18682b274a818393b757f49f4a51"
I20260924 21:17:20.970124 28747 raft_consensus.cc:3037] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.970081 28887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b80b002362bc451ab93f47a581fa8fe1" candidate_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2"
I20260924 21:17:20.970172 28887 raft_consensus.cc:3037] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:20.970728 28747 raft_consensus.cc:2454] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09cbcdb8a4764fcaa54c61e797ff6d40 in term 1.
I20260924 21:17:20.970898 28887 raft_consensus.cc:2454] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09cbcdb8a4764fcaa54c61e797ff6d40 in term 1.
I20260924 21:17:20.970901 28539 leader_election.cc:302] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09cbcdb8a4764fcaa54c61e797ff6d40, c37f18682b274a818393b757f49f4a51; no voters:
I20260924 21:17:20.970996 28980 raft_consensus.cc:2781] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:20.971103 28980 raft_consensus.cc:686] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 LEADER]: Becoming Leader. State: Replica: 09cbcdb8a4764fcaa54c61e797ff6d40, State: Running, Role: LEADER
I20260924 21:17:20.971175 28980 consensus_queue.cc:237] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } }
I20260924 21:17:20.971745 28448 catalog_manager.cc:5495] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09cbcdb8a4764fcaa54c61e797ff6d40" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 } health_report { overall_health: UNKNOWN } } }
I20260924 21:17:20.998710 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:20.996718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58380 (local address 127.20.115.193:36901)
0924 21:17:20.996810 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:20.996813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:20.996821 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:20.996844 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:20.996845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:20.996856 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:20.996915 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:20.997022 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.997483 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.998185 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:20.998371 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:20.998428 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:20.998528 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:20.998553 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:20.998587 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:20.998640 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 21:17:20.998655 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 21:17:21.003444 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:21.001074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40623 (local address 127.20.115.193:50873)
0924 21:17:21.001284 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:21.001293 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:21.001345 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:21.001502 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:21.001502 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:21.001503 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:17:21.001544 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:21.001546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:21.002139 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:21.002139 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:21.002683 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:21.002685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:21.003053 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:21.003054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:21.003094 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:21.003107 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:17:21.003314 (+ 207us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:17:21.003315 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:17:21.003346 (+ 31us) client_negotiation.cc:240] Negotiation successful
0924 21:17:21.003384 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":115,"threads_started":1}
I20260924 21:17:21.003451 28959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:21.001170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:50873 (local address 127.20.115.194:40623)
0924 21:17:21.001224 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:21.001226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:21.001355 (+ 129us) server_negotiation.cc:407] Connection header received
0924 21:17:21.001416 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:21.001417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:21.001427 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:21.001499 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:21.001615 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:21.002075 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:21.002775 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:21.002969 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:21.003028 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:21.003212 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:21.003245 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:21.003304 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:21.003360 (+ 56us) server_negotiation.cc:299] Negotiation successful
0924 21:17:21.003384 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:17:21.005489 28887 raft_consensus.cc:1261] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Refusing update from remote peer 09cbcdb8a4764fcaa54c61e797ff6d40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:21.005532 28747 raft_consensus.cc:1261] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Refusing update from remote peer 09cbcdb8a4764fcaa54c61e797ff6d40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:21.005751 29041 consensus_queue.cc:1035] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7ddadc9e6fb4233bfe2c9a0a52c97b2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:21.005837 28980 consensus_queue.cc:1035] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37f18682b274a818393b757f49f4a51" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:21.012676 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:21.010473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:57683 (local address 127.20.115.193:36901)
0924 21:17:21.010510 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:21.010512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:21.010519 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:17:21.010537 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:21.010538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:21.010546 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:21.010623 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:21.010719 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:21.011201 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:21.011937 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:21.012147 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:21.012222 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:21.012375 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:21.012430 (+ 55us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:21.012501 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:21.012570 (+ 69us) server_negotiation.cc:299] Negotiation successful
0924 21:17:21.012596 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:17:21.012698 29011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:21.010318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:36901 (local address 127.20.115.194:57683)
0924 21:17:21.010355 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:21.010366 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:21.010404 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:21.010626 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:21.010626 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:21.010627 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0924 21:17:21.010669 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:21.010670 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:21.011284 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:21.011285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:21.011852 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:21.011854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:21.012220 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:21.012221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:21.012276 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:21.012284 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0924 21:17:21.012511 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0924 21:17:21.012514 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:21.012566 (+ 52us) client_negotiation.cc:240] Negotiation successful
0924 21:17:21.012613 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260924 21:17:21.019059 28450 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:21.019158 28450 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39308:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:21.019905 28450 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 22f8cba9686b474b97f115f9873d083a: Sending DeleteTablet for 3 replicas of tablet b80b002362bc451ab93f47a581fa8fe1
I20260924 21:17:21.020174 28587 tablet_service.cc:1514] Processing DeleteTablet for tablet b80b002362bc451ab93f47a581fa8fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:21 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:58350
I20260924 21:17:21.020169 28727 tablet_service.cc:1514] Processing DeleteTablet for tablet b80b002362bc451ab93f47a581fa8fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:21 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:48882
I20260924 21:17:21.020301 28986 tablet_replica.cc:330] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40: stopping tablet replica
I20260924 21:17:21.020376 28986 raft_consensus.cc:2229] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:17:21.020435 28986 raft_consensus.cc:2258] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:21.020447 28867 tablet_service.cc:1514] Processing DeleteTablet for tablet b80b002362bc451ab93f47a581fa8fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:21 UTC) from {username='oryx', principal='oryx/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:50688
I20260924 21:17:21.020530 28985 tablet_replica.cc:330] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: stopping tablet replica
I20260924 21:17:21.020586 28985 raft_consensus.cc:2229] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:21.020622 28985 raft_consensus.cc:2258] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:21.020623 28984 tablet_replica.cc:330] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: stopping tablet replica
I20260924 21:17:21.020684 28984 raft_consensus.cc:2229] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:21.020720 28984 raft_consensus.cc:2258] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:21.020764 28986 ts_tablet_manager.cc:1905] T b80b002362bc451ab93f47a581fa8fe1 P 09cbcdb8a4764fcaa54c61e797ff6d40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:21.020941 28985 ts_tablet_manager.cc:1905] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:21.021181 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 28522
I20260924 21:17:21.022379 28984 ts_tablet_manager.cc:1905] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:21.022678 28985 ts_tablet_manager.cc:1918] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:17:21.022734 28985 log.cc:1196] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/b80b002362bc451ab93f47a581fa8fe1
I20260924 21:17:21.022938 28985 ts_tablet_manager.cc:1939] T b80b002362bc451ab93f47a581fa8fe1 P c37f18682b274a818393b757f49f4a51: Deleting consensus metadata
I20260924 21:17:21.023778 28435 catalog_manager.cc:4841] TS c37f18682b274a818393b757f49f4a51 (127.20.115.194:40623): tablet b80b002362bc451ab93f47a581fa8fe1 (table test-table [id=e2b4026faf814d788b48ee600c9c93ac]) successfully deleted
I20260924 21:17:21.024034 28984 ts_tablet_manager.cc:1918] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260924 21:17:21.024109 28984 log.cc:1196] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/b80b002362bc451ab93f47a581fa8fe1
I20260924 21:17:21.024310 28984 ts_tablet_manager.cc:1939] T b80b002362bc451ab93f47a581fa8fe1 P c7ddadc9e6fb4233bfe2c9a0a52c97b2: Deleting consensus metadata
I20260924 21:17:21.024680 28436 catalog_manager.cc:4841] TS c7ddadc9e6fb4233bfe2c9a0a52c97b2 (127.20.115.195:38831): tablet b80b002362bc451ab93f47a581fa8fe1 (table test-table [id=e2b4026faf814d788b48ee600c9c93ac]) successfully deleted
W20260924 21:17:21.028457 28437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:36901 (error 108)
I20260924 21:17:21.028784 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 28662
I20260924 21:17:21.030627 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:21.028889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:36901 (local address 127.0.0.1:58396)
0924 21:17:21.030527 (+ 1638us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:21.030578 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:36901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1617,"thread_start_us":40,"threads_started":1}
W20260924 21:17:21.032894 28437 catalog_manager.cc:4584] TS 09cbcdb8a4764fcaa54c61e797ff6d40 (127.20.115.193:36901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b80b002362bc451ab93f47a581fa8fe1: Network error: Client connection negotiation failed: client connection to 127.20.115.193:36901: connect: Connection refused (error 111)
I20260924 21:17:21.034883 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 28802
I20260924 21:17:21.040462 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 28418
2026-09-24T21:17:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2740 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:21 dist-test-slave-cxpd krb5kdc[29053](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:21 dist-test-slave-cxpd krb5kdc[29053](info): set up 2 sockets
Sep 24 21:17:21 dist-test-slave-cxpd krb5kdc[29053](info): commencing operation
krb5kdc: starting...
W20260924 21:17:22.069167 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:22 dist-test-slave-cxpd krb5kdc[29053](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:17:22.185700 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46787
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:46787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:22.264070 29066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:22.264264 29066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:22.264286 29066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:22.265439 29066 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:22.265480 29066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:22.265494 29066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:22.265506 29066 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:22.265519 29066 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:22.266975 29066 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:46787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:22.267217 29066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:22.267467 29066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:17:22.272459 29066 server_base.cc:1034] running on GCE node
W20260924 21:17:22.272418 29071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:22.272425 29072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:22.272439 29074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:22.272886 29066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:22.273196 29066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:22.274343 29066 hybrid_clock.cc:648] HybridClock initialized: now 1790284642274334 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-24T21:17:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1240 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:22 dist-test-slave-cxpd krb5kdc[29079](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:22 dist-test-slave-cxpd krb5kdc[29079](info): set up 2 sockets
Sep 24 21:17:22 dist-test-slave-cxpd krb5kdc[29079](info): commencing operation
krb5kdc: starting...
W20260924 21:17:23.317173 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:23 dist-test-slave-cxpd krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:17:23.433640 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:36839
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:36839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:23.512581 29092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:23.512835 29092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:23.512902 29092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:23.514007 29092 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:23.514043 29092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:23.514057 29092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:23.514070 29092 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:23.514081 29092 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:23.515440 29092 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:36839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:36839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:23.515614 29092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:23.515766 29092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:23.520481 29097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:23.520474 29098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:23.520502 29100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:23.520498 29092 server_base.cc:1034] running on GCE node
I20260924 21:17:23.521000 29092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:23.521303 29092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:23.522516 29092 hybrid_clock.cc:648] HybridClock initialized: now 1790284643522500 us; error 28 us; skew 500 ppm
Sep 24 21:17:23 dist-test-slave-cxpd krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284643, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:23.531234 29092 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:17:23.531787 29092 webserver.cc:460] Webserver started at http://127.20.115.254:35285/ using document root <none> and password file <none>
I20260924 21:17:23.531972 29092 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:23.532013 29092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:23.532100 29092 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:23.533188 29092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "82b25f3430374d10a06574de31dc89fa"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "60ce850563266339d3fe71b06af87966"
server_key_iv: "664e0d222af3d84382566da2723a5e81"
server_key_version: "encryptionkey@0"
I20260924 21:17:23.533494 29092 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "82b25f3430374d10a06574de31dc89fa"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "60ce850563266339d3fe71b06af87966"
server_key_iv: "664e0d222af3d84382566da2723a5e81"
server_key_version: "encryptionkey@0"
I20260924 21:17:23.534920 29092 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:23.535604 29107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:23.535812 29092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:23.535904 29092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "82b25f3430374d10a06574de31dc89fa"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "60ce850563266339d3fe71b06af87966"
server_key_iv: "664e0d222af3d84382566da2723a5e81"
server_key_version: "encryptionkey@0"
I20260924 21:17:23.535975 29092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:23.556815 29092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:23.558511 29092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:23.558656 29092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:23.562530 29092 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:36839
I20260924 21:17:23.562587 29159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:36839 every 8 connection(s)
I20260924 21:17:23.562891 29092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:23.563690 29160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:23.565837 29160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: Bootstrap starting.
I20260924 21:17:23.566416 29160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:23.566700 29160 log.cc:824] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:23.567356 29160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: No bootstrap required, opened a new log
I20260924 21:17:23.567924 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 29092
I20260924 21:17:23.568010 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:23.568214 20943 external_mini_cluster.cc:1428] Setting key 4ae4af2f490c4913f9d45b9a40d2534c
I20260924 21:17:23.569406 29160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } }
I20260924 21:17:23.569600 29160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:23.569643 29160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b25f3430374d10a06574de31dc89fa, State: Initialized, Role: FOLLOWER
I20260924 21:17:23.569809 29160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } }
I20260924 21:17:23.569927 29160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:23.570010 29160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:23.570072 29160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:23.570807 29160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } }
I20260924 21:17:23.570917 29160 leader_election.cc:302] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82b25f3430374d10a06574de31dc89fa; no voters:
I20260924 21:17:23.571113 29160 leader_election.cc:288] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:23.571141 29165 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:23.571245 29165 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 1 LEADER]: Becoming Leader. State: Replica: 82b25f3430374d10a06574de31dc89fa, State: Running, Role: LEADER
I20260924 21:17:23.571362 29160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:23.571378 29165 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } }
I20260924 21:17:23.571699 29166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82b25f3430374d10a06574de31dc89fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } } }
I20260924 21:17:23.571882 29166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:23.571687 29165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82b25f3430374d10a06574de31dc89fa. Latest consensus state: current_term: 1 leader_uuid: "82b25f3430374d10a06574de31dc89fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } } }
I20260924 21:17:23.572269 29165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:23.572451 29174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:23.572971 29174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:23.574626 29174 catalog_manager.cc:1347] Generated new cluster ID: 706f30d19a7c4a11b2fb28022979a91b
I20260924 21:17:23.574679 29174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:17:23 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:23.580399 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:23.568695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36156 (local address 127.20.115.254:36839)
0924 21:17:23.568868 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:23.568875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:23.568898 (+ 23us) server_negotiation.cc:407] Connection header received
0924 21:17:23.569645 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:23.569656 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:23.569725 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:23.569872 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:23.570924 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.572318 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.573391 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.573921 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.574056 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:23.576978 (+ 2922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:23.576981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:23.576981 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:23.576991 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:23.578846 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:23.579257 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:23.579260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:23.579260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:23.579356 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:23.579583 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:23.579584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:23.579585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:23.579850 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:23.579986 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:23.580099 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 21:17:23.580219 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":100,"threads_started":1}
I20260924 21:17:23.599100 29174 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:23.599617 29174 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:23.605713 29174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: Generated new TSK 0
I20260924 21:17:23.605892 29174 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:17:23.632069 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36839
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:23.711081 29186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:23.711263 29186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:23.711285 29186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:23.712447 29186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:23.712489 29186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:23.712515 29186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:23.713901 29186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:23.714123 29186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:23.714353 29186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:17:23.719190 29186 server_base.cc:1034] running on GCE node
W20260924 21:17:23.719173 29192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:23.719161 29194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:23.719165 29191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:23.719534 29186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:23.719772 29186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:23.720935 29186 hybrid_clock.cc:648] HybridClock initialized: now 1790284643720917 us; error 35 us; skew 500 ppm
Sep 24 21:17:23 dist-test-slave-cxpd krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284643, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:23.723261 29186 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:17:23.723673 29186 webserver.cc:460] Webserver started at http://127.20.115.193:34139/ using document root <none> and password file <none>
I20260924 21:17:23.723901 29186 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:23.723968 29186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:23.724120 29186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:23.725180 29186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "7d0e122bbec8450d9b5785ddef8d1067"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "498feba5942410252c851ddc420aa3ff"
server_key_iv: "1a82e4c655edf86be15553885f2c7af0"
server_key_version: "encryptionkey@0"
I20260924 21:17:23.725531 29186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "7d0e122bbec8450d9b5785ddef8d1067"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "498feba5942410252c851ddc420aa3ff"
server_key_iv: "1a82e4c655edf86be15553885f2c7af0"
server_key_version: "encryptionkey@0"
I20260924 21:17:23.726795 29186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:23.727478 29201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:23.727645 29186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:23.727733 29186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "7d0e122bbec8450d9b5785ddef8d1067"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "498feba5942410252c851ddc420aa3ff"
server_key_iv: "1a82e4c655edf86be15553885f2c7af0"
server_key_version: "encryptionkey@0"
I20260924 21:17:23.727813 29186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:23.741688 29186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:23.743371 29186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:23.743526 29186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:23.743736 29186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:23.744020 29186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:23.744073 29186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:23.744113 29186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:23.744140 29186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:23.749883 29186 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:43857
I20260924 21:17:23.749933 29314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:43857 every 8 connection(s)
I20260924 21:17:23.750208 29186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:17:23 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284643, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:23.756680 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 29186
I20260924 21:17:23.756822 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:23.757076 20943 external_mini_cluster.cc:1428] Setting key 63a5c18fbe0e3a0f06af37f6682089d5
I20260924 21:17:23.759435 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:23.750878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:48717 (local address 127.20.115.254:36839)
0924 21:17:23.750935 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:23.750936 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:23.751151 (+ 215us) server_negotiation.cc:407] Connection header received
0924 21:17:23.751658 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:23.751659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:23.751672 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:23.751722 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:23.752624 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.753216 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.754176 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.754409 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.754490 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:23.756424 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:23.756425 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:23.756425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:23.756433 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:23.758185 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:23.758541 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:23.758542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:23.758543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:23.758605 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:23.758823 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:23.758823 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:23.758824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:23.758967 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:23.759035 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:23.759266 (+ 231us) server_negotiation.cc:299] Negotiation successful
0924 21:17:23.759348 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:17:23.759625 29317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:23.750838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:48717)
0924 21:17:23.751039 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:23.751070 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:23.751549 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:23.751786 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:23.751789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:23.752333 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:23.752519 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:23.752523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.753306 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.753307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:23.754084 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:23.754086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.754499 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.754501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:23.754624 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:23.755060 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:23.755068 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:23.756327 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:23.758256 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:23.758261 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:23.758262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:23.758461 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:23.758668 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:23.758669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:23.758669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:23.758760 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:23.759028 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:23.759031 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:23.759131 (+ 100us) client_negotiation.cc:769] Sending connection context
0924 21:17:23.759244 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:17:23.759386 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":84,"threads_started":1}
I20260924 21:17:23.760205 29315 heartbeater.cc:347] Connected to a master server at 127.20.115.254:36839
I20260924 21:17:23.760277 29315 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:23.760454 29315 heartbeater.cc:511] Master 127.20.115.254:36839 requested a full tablet report, sending...
I20260924 21:17:23.760989 29124 ts_manager.cc:194] Registered new tserver with Master: 7d0e122bbec8450d9b5785ddef8d1067 (127.20.115.193:43857)
I20260924 21:17:23.761581 29124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:48717
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:17:23.781847 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36839
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:23.860345 29320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:23.860538 29320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:23.860559 29320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:23.861725 29320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:23.861768 29320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:23.861794 29320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:23.863119 29320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:23.863299 29320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:23.863461 29320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:23.868265 29328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:23.868254 29326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:23.868268 29325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:23.868544 29320 server_base.cc:1034] running on GCE node
I20260924 21:17:23.868742 29320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:23.869010 29320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:23.870174 29320 hybrid_clock.cc:648] HybridClock initialized: now 1790284643870162 us; error 23 us; skew 500 ppm
Sep 24 21:17:23 dist-test-slave-cxpd krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284643, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:23.872665 29320 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:17:23.873147 29320 webserver.cc:460] Webserver started at http://127.20.115.194:33949/ using document root <none> and password file <none>
I20260924 21:17:23.873375 29320 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:23.873422 29320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:23.873559 29320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:23.874526 29320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "99f799d0b7a14a5fb8b342fce0d844b6"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "b5c0541fe98fc6e56b963204bd07be77"
server_key_iv: "fc0a405a1b61ced4c7a3d17f4995dd58"
server_key_version: "encryptionkey@0"
I20260924 21:17:23.874874 29320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "99f799d0b7a14a5fb8b342fce0d844b6"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "b5c0541fe98fc6e56b963204bd07be77"
server_key_iv: "fc0a405a1b61ced4c7a3d17f4995dd58"
server_key_version: "encryptionkey@0"
I20260924 21:17:23.876128 29320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:23.876847 29335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:23.877022 29320 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:23.877116 29320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "99f799d0b7a14a5fb8b342fce0d844b6"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "b5c0541fe98fc6e56b963204bd07be77"
server_key_iv: "fc0a405a1b61ced4c7a3d17f4995dd58"
server_key_version: "encryptionkey@0"
I20260924 21:17:23.877198 29320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:23.893110 29320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:23.894783 29320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:23.894977 29320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:23.895182 29320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:23.895473 29320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:23.895526 29320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:23.895566 29320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:23.895593 29320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:23.901273 29320 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:41727
I20260924 21:17:23.901327 29448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:41727 every 8 connection(s)
I20260924 21:17:23.901608 29320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:17:23.906133 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 29320
I20260924 21:17:23.906256 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:23.906471 20943 external_mini_cluster.cc:1428] Setting key 9fea7e35c3a5eccf41bc182e972d945d
Sep 24 21:17:23 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284643, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:23.911206 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:23.902245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:51599 (local address 127.20.115.254:36839)
0924 21:17:23.902292 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:23.902295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:23.902488 (+ 193us) server_negotiation.cc:407] Connection header received
0924 21:17:23.902942 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:23.902943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:23.902953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:23.903024 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:23.903900 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.904407 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.905392 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.905573 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.905627 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:23.907902 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:23.907904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:23.907904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:23.907912 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:23.909659 (+ 1747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:23.910110 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:23.910111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:23.910112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:23.910175 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:23.910402 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:23.910403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:23.910403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:23.910559 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:23.910634 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:23.911014 (+ 380us) server_negotiation.cc:299] Negotiation successful
0924 21:17:23.911095 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:17:23.911234 29451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:23.902191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:51599)
0924 21:17:23.902346 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:23.902385 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:23.902850 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:23.903101 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:23.903104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:23.903639 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:23.903817 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:23.903821 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.904483 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.904483 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:23.905317 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:23.905319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:23.905646 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:23.905646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:23.905738 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:23.906173 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:23.906184 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:23.907809 (+ 1625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:23.909778 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:23.909784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:23.909785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:23.910030 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:23.910244 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:23.910244 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:23.910245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:23.910339 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:23.910649 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:23.910654 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:23.910864 (+ 210us) client_negotiation.cc:769] Sending connection context
0924 21:17:23.910963 (+ 99us) client_negotiation.cc:240] Negotiation successful
0924 21:17:23.911083 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260924 21:17:23.912074 29449 heartbeater.cc:347] Connected to a master server at 127.20.115.254:36839
I20260924 21:17:23.912184 29449 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:23.912384 29449 heartbeater.cc:511] Master 127.20.115.254:36839 requested a full tablet report, sending...
I20260924 21:17:23.912886 29124 ts_manager.cc:194] Registered new tserver with Master: 99f799d0b7a14a5fb8b342fce0d844b6 (127.20.115.194:41727)
I20260924 21:17:23.913287 29124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:51599
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:17:23.931740 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36839
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:24.010012 29454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:24.010236 29454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:24.010272 29454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:24.011402 29454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:24.011474 29454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:24.011520 29454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:24.012975 29454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:24.013222 29454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:24.013449 29454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:24.018348 29462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:24.018464 29454 server_base.cc:1034] running on GCE node
W20260924 21:17:24.018362 29460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:24.018388 29459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:24.018756 29454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:24.018977 29454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:24.020128 29454 hybrid_clock.cc:648] HybridClock initialized: now 1790284644020116 us; error 27 us; skew 500 ppm
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284644, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:24.022583 29454 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:17:24.023018 29454 webserver.cc:460] Webserver started at http://127.20.115.195:43147/ using document root <none> and password file <none>
I20260924 21:17:24.023236 29454 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:24.023303 29454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:24.023452 29454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:24.024475 29454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "4a03d869878747efa761ce4e1fdf3a2e"
format_stamp: "Formatted at 2026-09-24 21:17:24 on dist-test-slave-cxpd"
server_key: "ca37457b8b9e08f02a5b406e010193b3"
server_key_iv: "93d5c352026ba8a1cca7e00f1c8aa809"
server_key_version: "encryptionkey@0"
I20260924 21:17:24.024906 29454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "4a03d869878747efa761ce4e1fdf3a2e"
format_stamp: "Formatted at 2026-09-24 21:17:24 on dist-test-slave-cxpd"
server_key: "ca37457b8b9e08f02a5b406e010193b3"
server_key_iv: "93d5c352026ba8a1cca7e00f1c8aa809"
server_key_version: "encryptionkey@0"
I20260924 21:17:24.026257 29454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:24.026916 29469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.027086 29454 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:24.027181 29454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "4a03d869878747efa761ce4e1fdf3a2e"
format_stamp: "Formatted at 2026-09-24 21:17:24 on dist-test-slave-cxpd"
server_key: "ca37457b8b9e08f02a5b406e010193b3"
server_key_iv: "93d5c352026ba8a1cca7e00f1c8aa809"
server_key_version: "encryptionkey@0"
I20260924 21:17:24.027261 29454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:24.044826 29454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:24.047041 29454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:24.047206 29454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:24.047425 29454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:24.047725 29454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:24.047780 29454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.047816 29454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:24.047840 29454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.053565 29454 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:34115
I20260924 21:17:24.053637 29582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:34115 every 8 connection(s)
I20260924 21:17:24.053958 29454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:17:24.056134 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 29454
I20260924 21:17:24.056226 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:24.056447 20943 external_mini_cluster.cc:1428] Setting key e01d6f51a1b422da00716a442b2bb999
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284644, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:24.062743 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.054657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:52115 (local address 127.20.115.254:36839)
0924 21:17:24.054725 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.054728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.054916 (+ 188us) server_negotiation.cc:407] Connection header received
0924 21:17:24.055341 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.055342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.055352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.055408 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.056359 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.057129 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.058163 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.058342 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.058402 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.060310 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:24.060311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:24.060311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:24.060317 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:24.061616 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:24.061967 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:24.061968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:24.061969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:24.062025 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:24.062248 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:24.062249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:24.062249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:24.062359 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:24.062412 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:24.062623 (+ 211us) server_negotiation.cc:299] Negotiation successful
0924 21:17:24.062683 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:17:24.062876 29585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.054613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:52115)
0924 21:17:24.054772 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.054806 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.055263 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.055460 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.055463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.056003 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.056210 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.056216 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.057223 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.057223 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.058085 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.058087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.058415 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.058416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.058511 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.058904 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.058912 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.060211 (+ 1299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:24.061683 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:24.061688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:24.061690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:24.061885 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:24.062099 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:24.062099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:24.062100 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:24.062187 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:24.062412 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:24.062416 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:24.062518 (+ 102us) client_negotiation.cc:769] Sending connection context
0924 21:17:24.062595 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 21:17:24.062699 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260924 21:17:24.063279 29583 heartbeater.cc:347] Connected to a master server at 127.20.115.254:36839
I20260924 21:17:24.063364 29583 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:24.063542 29583 heartbeater.cc:511] Master 127.20.115.254:36839 requested a full tablet report, sending...
I20260924 21:17:24.063897 29124 ts_manager.cc:194] Registered new tserver with Master: 4a03d869878747efa761ce4e1fdf3a2e (127.20.115.195:34115)
I20260924 21:17:24.064216 29124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:52115
I20260924 21:17:24.068071 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab.
Entry for principal oryx/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab.
WARNING: no policy specified for oryx/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab.
Entry for principal oryx/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab.
WARNING: no policy specified for oryx/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab.
Entry for principal oryx/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab.
I20260924 21:17:24.140214 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 29186
I20260924 21:17:24.144770 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 29320
I20260924 21:17:24.148947 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 29454
I20260924 21:17:24.152995 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 29092
I20260924 21:17:24.157996 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:36839
--webserver_interface=127.20.115.254
--webserver_port=35285
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:36839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:24.237270 29592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:24.237466 29592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:24.237488 29592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:24.238623 29592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:24.238664 29592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:24.238678 29592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:24.238690 29592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:24.238703 29592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:24.240123 29592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:36839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:36839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=35285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:24.240350 29592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:24.240581 29592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:24.245330 29600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:24.245360 29597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:24.245360 29598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:24.245532 29592 server_base.cc:1034] running on GCE node
I20260924 21:17:24.245819 29592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:24.246115 29592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:24.247273 29592 hybrid_clock.cc:648] HybridClock initialized: now 1790284644247263 us; error 26 us; skew 500 ppm
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284644, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:24.249830 29592 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:17:24.250273 29592 webserver.cc:460] Webserver started at http://127.20.115.254:35285/ using document root <none> and password file <none>
I20260924 21:17:24.250514 29592 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:24.250574 29592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:24.252036 29592 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:24.252734 29607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.252942 29592 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:24.253053 29592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "82b25f3430374d10a06574de31dc89fa"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "60ce850563266339d3fe71b06af87966"
server_key_iv: "664e0d222af3d84382566da2723a5e81"
server_key_version: "encryptionkey@0"
I20260924 21:17:24.253402 29592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:24.263965 29592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:24.265895 29592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:24.266052 29592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:24.269856 29592 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:36839
I20260924 21:17:24.269956 29659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:36839 every 8 connection(s)
I20260924 21:17:24.270279 29592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:24.271746 29660 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:17:24.272403 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 29592
I20260924 21:17:24.272447 29660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: Bootstrap starting.
I20260924 21:17:24.272724 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:43857
--local_ip_for_outbound_sockets=127.20.115.193
--tserver_master_addrs=127.20.115.254:36839
--webserver_port=34139
--webserver_interface=127.20.115.193
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.115.193
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:17:24.281535 29660 log.cc:824] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:24.283180 29660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:17:24.283479 29660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: Bootstrap complete.
I20260924 21:17:24.285919 29660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } }
I20260924 21:17:24.286150 29660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b25f3430374d10a06574de31dc89fa, State: Initialized, Role: FOLLOWER
I20260924 21:17:24.286331 29660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } }
I20260924 21:17:24.286420 29660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:24.286477 29660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:24.286528 29660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:17:24.287535 29660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } }
I20260924 21:17:24.287621 29660 leader_election.cc:302] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82b25f3430374d10a06574de31dc89fa; no voters:
I20260924 21:17:24.287815 29660 leader_election.cc:288] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 21:17:24.287876 29664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:17:24.288028 29664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [term 2 LEADER]: Becoming Leader. State: Replica: 82b25f3430374d10a06574de31dc89fa, State: Running, Role: LEADER
I20260924 21:17:24.288132 29660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:24.288180 29664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } }
I20260924 21:17:24.288553 29665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "82b25f3430374d10a06574de31dc89fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } } }
I20260924 21:17:24.288632 29665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:24.288568 29666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82b25f3430374d10a06574de31dc89fa. Latest consensus state: current_term: 2 leader_uuid: "82b25f3430374d10a06574de31dc89fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82b25f3430374d10a06574de31dc89fa" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36839 } } }
I20260924 21:17:24.288677 29666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:24.290091 29680 catalog_manager.cc:1259] Loaded cluster ID: 706f30d19a7c4a11b2fb28022979a91b
I20260924 21:17:24.290145 29680 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: loading cluster ID for follower catalog manager: success
I20260924 21:17:24.291388 29680 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: acquiring CA information for follower catalog manager: success
I20260924 21:17:24.291742 29680 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:17:24.291817 29670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:24.291975 29670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:24.292055 29670 catalog_manager.cc:1259] Loaded cluster ID: 706f30d19a7c4a11b2fb28022979a91b
I20260924 21:17:24.292080 29670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:24.292246 29670 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:24.292327 29670 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 82b25f3430374d10a06574de31dc89fa: Loaded TSK: 0
I20260924 21:17:24.292478 29670 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 21:17:24.353905 29662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:24.354130 29662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:24.354153 29662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:24.355325 29662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:24.355365 29662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:24.355391 29662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:24.356884 29662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.193.keytab
--principal=oryx/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:43857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=34139
--tserver_master_addrs=127.20.115.254:36839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:24.357111 29662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:24.357355 29662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:24.362154 29686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:24.362154 29687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:24.362306 29689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:24.362444 29662 server_base.cc:1034] running on GCE node
I20260924 21:17:24.362676 29662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:24.362934 29662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:24.364091 29662 hybrid_clock.cc:648] HybridClock initialized: now 1790284644364073 us; error 33 us; skew 500 ppm
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284644, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:24.366542 29662 init.cc:377] Logged in from keytab as oryx/127.20.115.193@KRBTEST.COM (short username oryx)
I20260924 21:17:24.367018 29662 webserver.cc:460] Webserver started at http://127.20.115.193:34139/ using document root <none> and password file <none>
I20260924 21:17:24.367250 29662 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:24.367302 29662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:24.368628 29662 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:24.369288 29696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.369490 29662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:17:24.369575 29662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "7d0e122bbec8450d9b5785ddef8d1067"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "498feba5942410252c851ddc420aa3ff"
server_key_iv: "1a82e4c655edf86be15553885f2c7af0"
server_key_version: "encryptionkey@0"
I20260924 21:17:24.369908 29662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:24.382225 29662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:24.384284 29662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:24.384436 29662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:24.384645 29662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:24.384981 29662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:24.385046 29662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.385085 29662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:24.385111 29662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.391477 29662 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:43857
I20260924 21:17:24.391515 29809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:43857 every 8 connection(s)
I20260924 21:17:24.391906 29662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:17:24.397907 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 29662
I20260924 21:17:24.398248 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:41727
--local_ip_for_outbound_sockets=127.20.115.194
--tserver_master_addrs=127.20.115.254:36839
--webserver_port=33949
--webserver_interface=127.20.115.194
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.115.194
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:24.405341 29812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.392623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:42669)
0924 21:17:24.392841 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.392890 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.393554 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.397427 (+ 3873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.397431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.398173 (+ 742us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.398447 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.398454 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.400247 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.400248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.401116 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.401118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.401716 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.401718 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.401846 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.402383 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.402392 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.405143 (+ 2751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":111,"threads_started":1}
W20260924 21:17:24.405560 29810 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:36839 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.115.254:36839: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
W20260924 21:17:24.405717 29813 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:24.392751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:42669 (local address 127.20.115.254:36839)
0924 21:17:24.396971 (+ 4220us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.396981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.397025 (+ 44us) server_negotiation.cc:407] Connection header received
0924 21:17:24.397154 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.397162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.397218 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.397377 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.398598 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.400140 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.401218 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.401625 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.401744 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.405252 (+ 3508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:42669: BlockingRecv error: recv got EOF from 127.20.115.193:42669 (error 108)
Metrics: {"server-negotiator.queue_time_us":4165,"thread_start_us":93,"threads_started":1}
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:24.410048 29812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.405812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:50951)
0924 21:17:24.405970 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.405982 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.406036 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.406195 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.406196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.406322 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.406363 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.406364 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.406976 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.406977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.407481 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.407482 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.407835 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.407836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.407905 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.408314 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.408319 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.409964 (+ 1645us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141}
W20260924 21:17:24.410094 29813 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:24.405923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:50951 (local address 127.20.115.254:36839)
0924 21:17:24.405987 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.405989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.406021 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:17:24.406098 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.406099 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.406108 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.406176 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.406446 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.406898 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.407562 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.407752 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.407834 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.410034 (+ 2200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:50951: BlockingRecv error: recv got EOF from 127.20.115.193:50951 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:24.414836 29812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.410537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:40013)
0924 21:17:24.410659 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.410669 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.410718 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.410857 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.410857 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.410987 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.411011 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.411012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.411594 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.411595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.412075 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.412076 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.412481 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.412482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.412533 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.412959 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.412964 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.414751 (+ 1787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107}
W20260924 21:17:24.414973 29810 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:17:24.415083 29813 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:24.410619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:40013 (local address 127.20.115.254:36839)
0924 21:17:24.410654 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.410656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.410704 (+ 48us) server_negotiation.cc:407] Connection header received
0924 21:17:24.410775 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.410776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.410785 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.410854 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.411086 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.411518 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.412154 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.412386 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.412474 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.414827 (+ 2353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:40013: BlockingRecv error: recv got EOF from 127.20.115.193:40013 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 21:17:24.496637 29814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:24.496853 29814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:24.496876 29814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:24.497959 29814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:24.497991 29814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:24.498016 29814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:24.499315 29814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.194.keytab
--principal=oryx/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:41727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=33949
--tserver_master_addrs=127.20.115.254:36839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:24.499497 29814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:24.499661 29814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:24.504493 29822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:24.504531 29820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:24.504499 29819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:24.504834 29814 server_base.cc:1034] running on GCE node
I20260924 21:17:24.505035 29814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:24.505220 29814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:24.506376 29814 hybrid_clock.cc:648] HybridClock initialized: now 1790284644506351 us; error 37 us; skew 500 ppm
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284644, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:24.508872 29814 init.cc:377] Logged in from keytab as oryx/127.20.115.194@KRBTEST.COM (short username oryx)
I20260924 21:17:24.509253 29814 webserver.cc:460] Webserver started at http://127.20.115.194:33949/ using document root <none> and password file <none>
I20260924 21:17:24.509440 29814 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:24.509490 29814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:24.510835 29814 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:24.511417 29829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.511585 29814 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.511667 29814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "99f799d0b7a14a5fb8b342fce0d844b6"
format_stamp: "Formatted at 2026-09-24 21:17:23 on dist-test-slave-cxpd"
server_key: "b5c0541fe98fc6e56b963204bd07be77"
server_key_iv: "fc0a405a1b61ced4c7a3d17f4995dd58"
server_key_version: "encryptionkey@0"
I20260924 21:17:24.511978 29814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:24.527245 29814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:24.529445 29814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:24.529564 29814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:24.529765 29814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:24.530084 29814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:24.530118 29814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.530139 29814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:24.530189 29814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.537111 29814 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:41727
I20260924 21:17:24.537194 29942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:41727 every 8 connection(s)
I20260924 21:17:24.537530 29814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 21:17:24.544673 29813 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:24.538269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:59993 (local address 127.20.115.254:36839)
0924 21:17:24.538325 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.538328 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.538599 (+ 271us) server_negotiation.cc:407] Connection header received
0924 21:17:24.539103 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.539104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.539118 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.539171 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.540216 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.540700 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.541753 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.541934 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.541990 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.544626 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:59993: BlockingRecv error: recv got EOF from 127.20.115.194:59993 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:17:24.544677 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.538267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:59993)
0924 21:17:24.538462 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.538500 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.539014 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.539225 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.539228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.539792 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.540121 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.540128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.540860 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.540862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.541669 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.541671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.542010 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.542011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.542109 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.542545 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.542554 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.544539 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260924 21:17:24.544986 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 29814
W20260924 21:17:24.545135 29943 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:36839 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.115.254:36839: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
I20260924 21:17:24.545290 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:34115
--local_ip_for_outbound_sockets=127.20.115.195
--tserver_master_addrs=127.20.115.254:36839
--webserver_port=43147
--webserver_interface=127.20.115.195
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.115.195
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:24.549952 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.545385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:56205)
0924 21:17:24.545429 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.545439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.545476 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.545597 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.545597 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.545723 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.545762 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.545763 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.546394 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.546395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.547024 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.547027 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.547455 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.547456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.547520 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.547952 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.547956 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.549869 (+ 1913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 21:17:24.550009 29813 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:24.545438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:56205 (local address 127.20.115.254:36839)
0924 21:17:24.545486 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.545488 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.545498 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:24.545524 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.545525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.545534 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.545592 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.545838 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.546298 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.547120 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.547335 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.547382 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.549945 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:56205: BlockingRecv error: recv got EOF from 127.20.115.194:56205 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:24.554446 29945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.550234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:37077)
0924 21:17:24.550281 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.550291 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.550328 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.550484 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.550484 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.550610 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.550635 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.550637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.551248 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.551248 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.551742 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.551743 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.552089 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.552090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.552140 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.552556 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.552561 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.554366 (+ 1805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 21:17:24.554499 29813 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:24.550278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:37077 (local address 127.20.115.254:36839)
0924 21:17:24.550311 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.550313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.550321 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:24.550388 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.550389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.550398 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.550466 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.550707 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.551168 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.551828 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.552011 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.552092 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.554437 (+ 2345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:37077: BlockingRecv error: recv got EOF from 127.20.115.194:37077 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 21:17:24.554714 29943 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:17:24.626111 29946 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:24.626332 29946 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:24.626368 29946 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:24.627489 29946 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:24.627553 29946 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:24.627599 29946 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:24.629001 29946 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/krb5kdc/oryx_127.20.115.195.keytab
--principal=oryx/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:34115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=43147
--tserver_master_addrs=127.20.115.254:36839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29946
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:24.629256 29946 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:24.629514 29946 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:24.634268 29952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:24.634305 29954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:24.634284 29951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:24.634545 29946 server_base.cc:1034] running on GCE node
I20260924 21:17:24.634760 29946 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:24.635007 29946 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:24.636163 29946 hybrid_clock.cc:648] HybridClock initialized: now 1790284644636146 us; error 26 us; skew 500 ppm
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284644, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:24.638536 29946 init.cc:377] Logged in from keytab as oryx/127.20.115.195@KRBTEST.COM (short username oryx)
I20260924 21:17:24.638954 29946 webserver.cc:460] Webserver started at http://127.20.115.195:43147/ using document root <none> and password file <none>
I20260924 21:17:24.639183 29946 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:24.639264 29946 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:24.640511 29946 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:24.641054 29961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.641207 29946 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.641301 29946 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "4a03d869878747efa761ce4e1fdf3a2e"
format_stamp: "Formatted at 2026-09-24 21:17:24 on dist-test-slave-cxpd"
server_key: "ca37457b8b9e08f02a5b406e010193b3"
server_key_iv: "93d5c352026ba8a1cca7e00f1c8aa809"
server_key_version: "encryptionkey@0"
I20260924 21:17:24.641615 29946 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:24.656635 29946 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:24.658928 29946 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:24.659089 29946 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:24.659322 29946 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:24.659641 29946 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:24.659698 29946 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.659732 29946 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:24.659758 29946 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:24.665400 29946 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:34115
I20260924 21:17:24.665455 30074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:34115 every 8 connection(s)
I20260924 21:17:24.665756 29946 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:17:24.670195 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 29946
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 21:17:24.673578 29813 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:24.666448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:58177 (local address 127.20.115.254:36839)
0924 21:17:24.666519 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.666521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.666770 (+ 249us) server_negotiation.cc:407] Connection header received
0924 21:17:24.667249 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.667250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.667260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.667310 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.668175 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.668644 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.669755 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.669944 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.670000 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.673498 (+ 3498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:58177: BlockingRecv error: recv got EOF from 127.20.115.195:58177 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:17:24.673646 30077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.666488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:58177)
0924 21:17:24.666649 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.666682 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.667167 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.667360 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.667363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.667903 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.668092 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.668096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.668741 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.668742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.669672 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.669675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.670038 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.670040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.670200 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.670756 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.670766 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.673428 (+ 2662us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
W20260924 21:17:24.673889 30075 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:36839 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.115.254:36839: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
I20260924 21:17:24.679602 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.670762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36166 (local address 127.20.115.254:36839)
0924 21:17:24.670891 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.670893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.670902 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:24.670924 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.670925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.670936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.671013 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.671667 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.672186 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.673184 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.673492 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.673565 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.675151 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:24.675155 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:24.675156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:24.675167 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:24.677464 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:24.678009 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:24.678012 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:24.678013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:24.678124 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:24.678359 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:24.678360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:24.678360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:24.678766 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:24.678906 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:24.679357 (+ 451us) server_negotiation.cc:299] Negotiation successful
0924 21:17:24.679482 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:24.682205 30077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.674141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:36543)
0924 21:17:24.674204 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.674216 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.674256 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.677101 (+ 2845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.677102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.677255 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.677310 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.677312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.678020 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.678021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.678740 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.678743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.679596 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.679597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.679701 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.680077 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.680100 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.682133 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":16}
W20260924 21:17:24.682313 29813 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:24.676884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:36543 (local address 127.20.115.254:36839)
0924 21:17:24.676948 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.676950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.676958 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:24.676977 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.676978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.677003 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.677060 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.677403 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.677904 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.678994 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.679499 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.679585 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.682250 (+ 2665us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:36543: BlockingRecv error: recv got EOF from 127.20.115.195:36543 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:24 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:24.686757 30077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:24.682612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:49577)
0924 21:17:24.682651 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:24.682660 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:24.682694 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:24.682835 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:24.682835 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:24.682982 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:24.683013 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.683014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.683942 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.683943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.684432 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:24.684433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.684740 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.684740 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:24.684773 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.685248 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:24.685253 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:24.686716 (+ 1463us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260924 21:17:24.686851 29813 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:24.682655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:49577 (local address 127.20.115.254:36839)
0924 21:17:24.682726 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:24.682728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:24.682737 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:24.682757 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:24.682758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:24.682767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:24.682833 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:24.683109 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.683836 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.684503 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:24.684673 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:24.684729 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:24.686788 (+ 2059us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:49577: BlockingRecv error: recv got EOF from 127.20.115.195:49577 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260924 21:17:24.686903 30075 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 24 21:17:25 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:25 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:25.420145 30080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:25.415396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:53919)
0924 21:17:25.415636 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:25.415655 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:25.415719 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:25.415888 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:25.415889 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:25.416052 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:25.416101 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:25.416103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.416931 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.416932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:25.417492 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:25.417494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.417878 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.417880 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:25.417927 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:25.418381 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:25.418386 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:25.420079 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
W20260924 21:17:25.420243 30081 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:25.415499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:53919 (local address 127.20.115.254:36839)
0924 21:17:25.415662 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:25.415663 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:25.415720 (+ 57us) server_negotiation.cc:407] Connection header received
0924 21:17:25.415782 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:25.415782 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:25.415792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:25.415854 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:25.416175 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.416763 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.417579 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.417776 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.417827 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:25.420170 (+ 2343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:53919: BlockingRecv error: recv got EOF from 127.20.115.193:53919 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
Sep 24 21:17:25 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:25 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:25.560077 30082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:25.555090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:50545)
0924 21:17:25.555254 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:25.555265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:25.555328 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:25.555469 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:25.555470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:25.555697 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:25.555747 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:25.555749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.556400 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.556401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:25.557181 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:25.557183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.557550 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.557550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:25.557594 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:25.558127 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:25.558138 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:25.560005 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":66,"threads_started":1}
W20260924 21:17:25.560213 30081 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:25.555151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:50545 (local address 127.20.115.254:36839)
0924 21:17:25.555252 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:25.555254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:25.555311 (+ 57us) server_negotiation.cc:407] Connection header received
0924 21:17:25.555388 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:25.555388 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:25.555398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:25.555447 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:25.555809 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.556306 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.557261 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.557478 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.557542 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:25.560128 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:50545: BlockingRecv error: recv got EOF from 127.20.115.194:50545 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Sep 24 21:17:25 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:25 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:25.691910 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:25.687319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:59411)
0924 21:17:25.687496 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:25.687507 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:25.687573 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:25.687695 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:25.687696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:25.687849 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:25.687898 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:25.687899 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.688602 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.688603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:25.689266 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:25.689269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.689595 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.689596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:25.689639 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:25.690106 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:25.690110 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:25.691842 (+ 1732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
W20260924 21:17:25.692024 30081 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:25.687366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:59411 (local address 127.20.115.254:36839)
0924 21:17:25.687435 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:25.687438 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:25.687571 (+ 133us) server_negotiation.cc:407] Connection header received
0924 21:17:25.687616 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:25.687617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:25.687628 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:25.687677 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:25.687958 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.688534 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.689355 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:25.689535 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:25.689584 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:25.691912 (+ 2328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:59411: BlockingRecv error: recv got EOF from 127.20.115.195:59411 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Sep 24 21:17:26 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:26 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:26.425200 30084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:26.420635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:34465)
0924 21:17:26.420766 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:26.420798 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:26.420858 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:26.421049 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:26.421050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:26.421204 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:26.421251 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:26.421253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.421884 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.421885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:26.422415 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:26.422417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.422755 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.422755 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:26.422793 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:26.423231 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:26.423235 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:26.425104 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
W20260924 21:17:26.425302 30085 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:26.420722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:34465 (local address 127.20.115.254:36839)
0924 21:17:26.420873 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:26.420875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:26.420885 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:26.420913 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:26.420914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:26.420928 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:26.421035 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:26.421320 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.421815 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.422494 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.422690 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.422743 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:26.425218 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:34465: BlockingRecv error: recv got EOF from 127.20.115.193:34465 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
Sep 24 21:17:26 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:26 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:26.565145 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:26.560532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:38343)
0924 21:17:26.560657 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:26.560676 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:26.560730 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:26.560917 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:26.560918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:26.561096 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:26.561143 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:26.561146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.561827 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.561829 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:26.562370 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:26.562372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.562749 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.562750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:26.562806 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:26.563259 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:26.563264 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:26.565056 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
W20260924 21:17:26.565302 30085 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:26.560633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:38343 (local address 127.20.115.254:36839)
0924 21:17:26.560678 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:26.560680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:26.560715 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:17:26.560824 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:26.560825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:26.560845 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:26.560910 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:26.561233 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.561742 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.562470 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.562672 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.562752 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:26.565160 (+ 2408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:38343: BlockingRecv error: recv got EOF from 127.20.115.194:38343 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 24 21:17:26 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:26 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:26.696939 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:26.692349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:46199)
0924 21:17:26.692475 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:26.692493 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:26.692570 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:26.692735 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:26.692736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:26.692942 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:26.692988 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:26.692990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.693667 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.693669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:26.694200 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:26.694201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.694579 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.694580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:26.694646 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:26.695078 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:26.695082 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:26.696882 (+ 1800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
W20260924 21:17:26.697052 30085 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:26.692478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:46199 (local address 127.20.115.254:36839)
0924 21:17:26.692563 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:26.692565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:26.692575 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:26.692634 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:26.692635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:26.692649 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:26.692717 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:26.693075 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.693578 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.694303 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:26.694495 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:26.694578 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:26.696964 (+ 2386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:46199: BlockingRecv error: recv got EOF from 127.20.115.195:46199 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Sep 24 21:17:27 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:27 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:27.429943 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:27.425702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:51691)
0924 21:17:27.425840 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:27.425851 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:27.425899 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:27.426020 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:27.426020 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:27.426175 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:27.426223 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:27.426224 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.426850 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.426851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:27.427373 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:27.427374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.427692 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.427692 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:27.427729 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:27.428236 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:27.428241 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:27.429883 (+ 1642us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
W20260924 21:17:27.430078 30089 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:27.425762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:51691 (local address 127.20.115.254:36839)
0924 21:17:27.425898 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:27.425900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:27.425909 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:27.425945 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:27.425946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:27.425955 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:27.426008 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:27.426283 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.426785 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.427440 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.427632 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.427681 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:27.429983 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:51691: BlockingRecv error: recv got EOF from 127.20.115.193:51691 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":80,"threads_started":1}
Sep 24 21:17:27 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:27 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:27.570492 30090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:27.565623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:58317)
0924 21:17:27.565788 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:27.565802 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:27.565906 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:27.566065 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:27.566066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:27.566235 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:27.566280 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:27.566282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.566948 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.566949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:27.567481 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:27.567483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.567860 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.567861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:27.567924 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:27.568396 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:27.568401 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:27.570381 (+ 1980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
W20260924 21:17:27.570518 30089 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:27.565719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:58317 (local address 127.20.115.254:36839)
0924 21:17:27.565792 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:27.565794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:27.565882 (+ 88us) server_negotiation.cc:407] Connection header received
0924 21:17:27.565992 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:27.565994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:27.566006 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:27.566060 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:27.566356 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.566861 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.567568 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.567776 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.567859 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:27.570460 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:58317: BlockingRecv error: recv got EOF from 127.20.115.194:58317 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 24 21:17:27 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:27 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:27.701917 30091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:27.697395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:32803)
0924 21:17:27.697536 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:27.697550 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:27.697608 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:27.697736 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:27.697737 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:27.697893 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:27.697941 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:27.697942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.698554 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.698555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:27.699099 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:27.699100 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.699394 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.699395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:27.699431 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:27.699897 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:27.699903 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:27.701860 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
W20260924 21:17:27.702046 30089 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:27.697487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:32803 (local address 127.20.115.254:36839)
0924 21:17:27.697548 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:27.697550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:27.697607 (+ 57us) server_negotiation.cc:407] Connection header received
0924 21:17:27.697649 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:27.697649 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:27.697661 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:27.697719 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:27.698007 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.698485 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.699163 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:27.699333 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:27.699382 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:27.701942 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:32803: BlockingRecv error: recv got EOF from 127.20.115.195:32803 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Sep 24 21:17:28 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:28 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:28.435002 30092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:28.430465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:43655)
0924 21:17:28.430620 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:28.430630 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:28.430676 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:28.430819 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:28.430820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:28.430977 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:28.431028 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:28.431030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.431700 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.431701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:28.432226 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:28.432227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.432572 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.432572 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:28.432609 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:28.433072 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:28.433076 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:28.434945 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
W20260924 21:17:28.435170 30093 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:28.430544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:43655 (local address 127.20.115.254:36839)
0924 21:17:28.430675 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:28.430677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:28.430688 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:28.430725 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:28.430725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:28.430735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:28.430798 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:28.431108 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.431626 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.432305 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.432505 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.432570 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:28.435078 (+ 2508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:43655: BlockingRecv error: recv got EOF from 127.20.115.193:43655 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
Sep 24 21:17:28 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:28 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:28.581446 30094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:28.571057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:59675)
0924 21:17:28.571254 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:28.571265 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:28.571361 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:28.571535 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:28.571536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:28.571720 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:28.571767 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:28.571768 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.572507 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.572508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:28.573199 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:28.573201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.573584 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.573585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:28.573641 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:28.579361 (+ 5720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:28.579366 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:28.581386 (+ 2020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":112,"threads_started":1}
W20260924 21:17:28.581588 30093 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:28.571144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:59675 (local address 127.20.115.254:36839)
0924 21:17:28.571222 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:28.571225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:28.571341 (+ 116us) server_negotiation.cc:407] Connection header received
0924 21:17:28.571438 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:28.571439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:28.571460 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:28.571527 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:28.571858 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.572423 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.573281 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.573482 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.573586 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:28.581508 (+ 7922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:59675: BlockingRecv error: recv got EOF from 127.20.115.194:59675 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Sep 24 21:17:28 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:28 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:28.707243 30095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:28.702401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:45015)
0924 21:17:28.702571 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:28.702583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:28.702672 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:28.702859 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:28.702860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:28.703053 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:28.703100 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:28.703102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.703788 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.703789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:28.704562 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:28.704564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.705001 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.705002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:28.705059 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:28.705457 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:28.705461 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:28.707145 (+ 1684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
W20260924 21:17:28.707288 30093 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:28.702490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:45015 (local address 127.20.115.254:36839)
0924 21:17:28.702557 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:28.702560 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:28.702651 (+ 91us) server_negotiation.cc:407] Connection header received
0924 21:17:28.702752 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:28.702753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:28.702773 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:28.702851 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:28.703178 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.703673 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.704638 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:28.704916 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:28.705000 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:28.707225 (+ 2225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:45015: BlockingRecv error: recv got EOF from 127.20.115.195:45015 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 24 21:17:29 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:29 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:29.439976 30096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:29.435481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:47615)
0924 21:17:29.435645 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:29.435656 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:29.435700 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:29.435849 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:29.435850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:29.435998 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:29.436047 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:29.436048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.436681 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.436681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:29.437257 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:29.437258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.437591 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.437591 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:29.437630 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:29.438163 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:29.438168 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:29.439880 (+ 1712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
W20260924 21:17:29.440024 30097 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:29.435578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:47615 (local address 127.20.115.254:36839)
0924 21:17:29.435720 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:29.435722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:29.435730 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:29.435753 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:29.435753 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:29.435764 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:29.435832 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:29.436110 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.436618 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.437330 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.437524 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.437582 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:29.439956 (+ 2374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:47615: BlockingRecv error: recv got EOF from 127.20.115.193:47615 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
Sep 24 21:17:29 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:29 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:29.586593 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:29.581931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:52063)
0924 21:17:29.582073 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:29.582085 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:29.582143 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:29.582401 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:29.582402 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:29.582571 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:29.582618 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:29.582620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.583310 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.583310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:29.583864 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:29.583865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.584207 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.584208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:29.584251 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:29.584723 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:29.584727 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:29.586536 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":56,"threads_started":1}
W20260924 21:17:29.586747 30097 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:29.582088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:52063 (local address 127.20.115.254:36839)
0924 21:17:29.582256 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:29.582258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:29.582266 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:29.582290 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:29.582291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:29.582301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:29.582357 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:29.582697 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.583231 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.583953 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.584140 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.584195 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:29.586656 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:52063: BlockingRecv error: recv got EOF from 127.20.115.194:52063 (error 108)
Metrics: {"server-negotiator.queue_time_us":138}
Sep 24 21:17:29 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:29 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:29.712589 30099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:29.707764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:39937)
0924 21:17:29.707936 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:29.707946 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:29.708007 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:29.708154 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:29.708155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:29.708424 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:29.708485 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:29.708487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.709201 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.709203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:29.709738 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:29.709740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.710102 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.710103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:29.710155 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:29.710596 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:29.710600 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:29.712489 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
W20260924 21:17:29.712620 30097 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:29.707832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:39937 (local address 127.20.115.254:36839)
0924 21:17:29.707945 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:29.707947 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:29.707999 (+ 52us) server_negotiation.cc:407] Connection header received
0924 21:17:29.708056 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:29.708057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:29.708073 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:29.708137 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:29.708576 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.709091 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.709833 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:29.710008 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:29.710072 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:29.712562 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:39937: BlockingRecv error: recv got EOF from 127.20.115.195:39937 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Sep 24 21:17:30 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:30 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:30.445287 30100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:30.440476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:34625)
0924 21:17:30.440629 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:30.440639 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:30.440724 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:30.440954 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:30.440955 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:30.441171 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:30.441231 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:30.441233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.441892 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.441893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:30.442437 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:30.442438 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.442760 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.442760 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:30.442799 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:30.443227 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:30.443232 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:30.445191 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
W20260924 21:17:30.445343 30101 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:30.440576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:34625 (local address 127.20.115.254:36839)
0924 21:17:30.440690 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:30.440692 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:30.440702 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:30.440839 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:30.440840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:30.440851 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:30.440931 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:30.441317 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.441811 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.442505 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.442697 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.442747 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:30.445270 (+ 2523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:34625: BlockingRecv error: recv got EOF from 127.20.115.193:34625 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
Sep 24 21:17:30 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:30 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:30.592146 30102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:30.587101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:59045)
0924 21:17:30.587233 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:30.587244 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:30.587301 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:30.587479 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:30.587481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:30.587737 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:30.587786 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:30.587788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.588488 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.588489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:30.589222 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:30.589224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.589603 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.589604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:30.589647 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:30.590066 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:30.590070 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:30.592073 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":78,"threads_started":1}
W20260924 21:17:30.592187 30101 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:30.587177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:59045 (local address 127.20.115.254:36839)
0924 21:17:30.587245 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:30.587247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:30.587286 (+ 39us) server_negotiation.cc:407] Connection header received
0924 21:17:30.587365 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:30.587366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:30.587377 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:30.587435 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:30.587894 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.588392 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.589316 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.589533 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.589593 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:30.592140 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:59045: BlockingRecv error: recv got EOF from 127.20.115.194:59045 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 24 21:17:30 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:30 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:30.717650 30103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:30.713033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:56965)
0924 21:17:30.713155 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:30.713169 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:30.713227 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:30.713360 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:30.713361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:30.713510 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:30.713558 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:30.713560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.714342 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.714344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:30.714934 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:30.714937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.715279 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.715280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:30.715319 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:30.715815 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:30.715819 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:30.717594 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
W20260924 21:17:30.717782 30101 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:30.713109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:56965 (local address 127.20.115.254:36839)
0924 21:17:30.713158 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:30.713160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:30.713212 (+ 52us) server_negotiation.cc:407] Connection header received
0924 21:17:30.713278 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:30.713279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:30.713290 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:30.713346 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:30.713619 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.714133 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.715023 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:30.715209 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:30.715269 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:30.717700 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:56965: BlockingRecv error: recv got EOF from 127.20.115.195:56965 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Sep 24 21:17:31 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:31 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:31.450759 30104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:31.445768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:40363)
0924 21:17:31.445916 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:31.445926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:31.445974 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:31.446131 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:31.446132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:31.446280 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:31.446331 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:31.446333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.447014 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.447014 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:31.447680 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:31.447682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.448109 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.448110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:31.448151 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:31.448607 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:31.448612 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:31.450653 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":89,"threads_started":1}
W20260924 21:17:31.450816 30105 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:31.445862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:40363 (local address 127.20.115.254:36839)
0924 21:17:31.446012 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:31.446014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:31.446021 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:17:31.446043 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:31.446043 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:31.446053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:31.446115 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:31.446432 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.446944 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.447741 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.448040 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.448098 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:31.450736 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:40363: BlockingRecv error: recv got EOF from 127.20.115.193:40363 (error 108)
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
Sep 24 21:17:31 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:31 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:31.597337 30106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:31.592630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:52099)
0924 21:17:31.592813 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:31.592825 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:31.592885 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:31.593027 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:31.593028 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:31.593193 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:31.593242 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:31.593244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.593928 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.593928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:31.594461 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:31.594462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.594833 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.594834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:31.594885 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:31.595301 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:31.595306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:31.597275 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
W20260924 21:17:31.597486 30105 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:31.592726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:52099 (local address 127.20.115.254:36839)
0924 21:17:31.592868 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:31.592869 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:31.592879 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:31.592946 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:31.592947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:31.592958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:31.593012 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:31.593327 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.593831 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.594533 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.594718 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.594775 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:31.597397 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:52099: BlockingRecv error: recv got EOF from 127.20.115.194:52099 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 24 21:17:31 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:31 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:31.723100 30107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:31.718130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:39709)
0924 21:17:31.718293 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:31.718304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:31.718364 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:31.718493 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:31.718494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:31.718665 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:31.718725 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:31.718727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.719448 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.719448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:31.719972 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:31.719973 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.720360 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.720361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:31.720417 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:31.721083 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:31.721089 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:31.722993 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
W20260924 21:17:31.723152 30105 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:31.718200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:39709 (local address 127.20.115.254:36839)
0924 21:17:31.718270 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:31.718272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:31.718362 (+ 90us) server_negotiation.cc:407] Connection header received
0924 21:17:31.718421 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:31.718421 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:31.718433 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:31.718474 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:31.718822 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.719367 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.720051 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:31.720263 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:31.720329 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:31.723081 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:39709: BlockingRecv error: recv got EOF from 127.20.115.195:39709 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 24 21:17:32 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:32 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:32.455948 30108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:32.451188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:43141)
0924 21:17:32.451372 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:32.451384 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:32.451438 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:32.451584 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:32.451585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:32.451730 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:32.451794 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:32.451796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.452425 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.452426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:32.453101 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:32.453103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.453424 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.453425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:32.453468 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:32.453889 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:32.453893 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:32.455881 (+ 1988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
W20260924 21:17:32.456087 30109 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:32.451306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:43141 (local address 127.20.115.254:36839)
0924 21:17:32.451421 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:32.451423 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:32.451430 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:17:32.451494 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:32.451494 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:32.451504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:32.451562 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:32.451866 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.452354 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.453173 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.453361 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.453414 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:32.455972 (+ 2558us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:43141: BlockingRecv error: recv got EOF from 127.20.115.193:43141 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":54,"threads_started":1}
Sep 24 21:17:32 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:32 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:32.602523 30110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:32.597869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:55507)
0924 21:17:32.598027 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:32.598041 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:32.598098 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:32.598234 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:32.598234 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:32.598395 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:32.598444 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:32.598446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.599286 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.599286 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:32.599795 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:32.599796 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.600110 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.600111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:32.600150 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:32.600724 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:32.600730 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:32.602460 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
W20260924 21:17:32.602661 30109 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:32.597946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:55507 (local address 127.20.115.254:36839)
0924 21:17:32.598029 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:32.598031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:32.598100 (+ 69us) server_negotiation.cc:407] Connection header received
0924 21:17:32.598144 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:32.598145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:32.598156 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:32.598211 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:32.598505 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.599217 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.599865 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.600047 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.600098 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:32.602578 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:55507: BlockingRecv error: recv got EOF from 127.20.115.194:55507 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 24 21:17:32 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:32 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:32.728338 30111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:32.723586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:47191)
0924 21:17:32.723759 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:32.723772 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:32.723842 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:32.724014 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:32.724015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:32.724194 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:32.724241 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:32.724243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.724971 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.724973 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:32.725515 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:32.725516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.725893 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.725894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:32.725951 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:32.726409 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:32.726413 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:32.728231 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":94,"threads_started":1}
W20260924 21:17:32.728375 30109 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:32.723670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:47191 (local address 127.20.115.254:36839)
0924 21:17:32.723759 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:32.723761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:32.723823 (+ 62us) server_negotiation.cc:407] Connection header received
0924 21:17:32.723896 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:32.723897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:32.723911 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:32.724004 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:32.724325 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.724894 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.725613 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:32.725810 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:32.725892 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:32.728313 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:47191: BlockingRecv error: recv got EOF from 127.20.115.195:47191 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 24 21:17:33 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:33 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:33.461838 30112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:33.456611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:40905)
0924 21:17:33.456751 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:33.456762 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:33.456879 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:33.457007 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:33.457008 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:33.457173 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:33.457224 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:33.457225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.457943 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.457944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:33.458478 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:33.458480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.458856 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.458857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:33.458945 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:33.459479 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:33.459486 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:33.461738 (+ 2252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
W20260924 21:17:33.462011 30113 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:33.456699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:40905 (local address 127.20.115.254:36839)
0924 21:17:33.456877 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:33.456879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:33.456889 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:33.456932 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:33.456932 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:33.456943 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:33.457004 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:33.457315 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.457856 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.458561 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.458756 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.458851 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:33.461825 (+ 2974us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:40905: BlockingRecv error: recv got EOF from 127.20.115.193:40905 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":55,"threads_started":1}
Sep 24 21:17:33 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:33 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:33.611505 30114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:33.603063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:36439)
0924 21:17:33.603213 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:33.603226 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:33.603303 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:33.603509 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:33.603510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:33.603716 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:33.603763 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:33.603765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.604405 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.604406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:33.605079 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:33.605082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.605420 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.605421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:33.605463 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:33.609227 (+ 3764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:33.609233 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:33.611367 (+ 2134us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
W20260924 21:17:33.611539 30113 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:33.603110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:36439 (local address 127.20.115.254:36839)
0924 21:17:33.603215 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:33.603217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:33.603312 (+ 95us) server_negotiation.cc:407] Connection header received
0924 21:17:33.603418 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:33.603419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:33.603434 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:33.603498 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:33.603834 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.604332 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.605166 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.605353 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.605408 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:33.611472 (+ 6064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:36439: BlockingRecv error: recv got EOF from 127.20.115.194:36439 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 24 21:17:33 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:33 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:33.733649 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:33.728931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:49437)
0924 21:17:33.729126 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:33.729141 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:33.729202 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:33.729365 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:33.729365 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:33.729535 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:33.729584 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:33.729586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.730277 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.730279 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:33.730821 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:33.730823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.731219 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.731220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:33.731261 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:33.731712 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:33.731717 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:33.733587 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":140,"threads_started":1}
W20260924 21:17:33.733803 30113 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:33.729008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:49437 (local address 127.20.115.254:36839)
0924 21:17:33.729108 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:33.729110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:33.729238 (+ 128us) server_negotiation.cc:407] Connection header received
0924 21:17:33.729268 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:33.729268 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:33.729283 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:33.729346 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:33.729663 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.730178 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.730930 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:33.731145 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:33.731209 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:33.733722 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:49437: BlockingRecv error: recv got EOF from 127.20.115.195:49437 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Sep 24 21:17:34 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:34 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:34.467270 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:34.462399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:47577)
0924 21:17:34.462565 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:34.462576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:34.462626 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:34.462782 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:34.462783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:34.462951 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:34.463012 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:34.463013 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.463750 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.463751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:34.464312 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:34.464314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.464685 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.464686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:34.464749 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:34.465392 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:34.465398 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:34.467212 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
W20260924 21:17:34.467423 30117 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:34.462467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:47577 (local address 127.20.115.254:36839)
0924 21:17:34.462604 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:34.462605 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:34.462615 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:34.462687 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:34.462688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:34.462698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:34.462765 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:34.463085 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.463654 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.464399 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.464589 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.464643 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:34.467309 (+ 2666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:47577: BlockingRecv error: recv got EOF from 127.20.115.193:47577 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":74,"threads_started":1}
Sep 24 21:17:34 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:34 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:34.616696 30118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:34.612036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:53035)
0924 21:17:34.612254 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:34.612267 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:34.612352 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:34.612508 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:34.612508 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:34.612670 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:34.612719 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:34.612721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.613394 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.613395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:34.613924 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:34.613925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.614238 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.614239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:34.614277 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:34.614740 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:34.614744 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:34.616638 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":161,"threads_started":1}
W20260924 21:17:34.616858 30117 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:34.612104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:53035 (local address 127.20.115.254:36839)
0924 21:17:34.612194 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:34.612196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:34.612341 (+ 145us) server_negotiation.cc:407] Connection header received
0924 21:17:34.612423 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:34.612424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:34.612437 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:34.612487 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:34.612824 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.613320 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.613996 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.614177 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.614226 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:34.616736 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:53035: BlockingRecv error: recv got EOF from 127.20.115.194:53035 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 24 21:17:34 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:34 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:34.738598 30119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:34.734089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:51581)
0924 21:17:34.734207 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:34.734225 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:34.734296 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:34.734442 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:34.734443 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:34.734617 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:34.734663 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:34.734664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.735322 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.735323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:34.735837 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:34.735838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.736193 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.736195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:34.736255 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:34.736737 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:34.736741 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:34.738503 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":85,"threads_started":1}
W20260924 21:17:34.738651 30117 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:34.734223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:51581 (local address 127.20.115.254:36839)
0924 21:17:34.734294 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:34.734296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:34.734306 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:17:34.734354 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:34.734355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:34.734375 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:34.734436 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:34.734739 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.735241 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.735926 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:34.736114 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:34.736184 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:34.738591 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:51581: BlockingRecv error: recv got EOF from 127.20.115.195:51581 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 24 21:17:35 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:35 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:35.472327 30120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:35.467807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:45847)
0924 21:17:35.467970 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:35.467982 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:35.468036 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:35.468165 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:35.468166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:35.468327 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:35.468374 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:35.468375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.469069 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.469070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:35.469631 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:35.469633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.469968 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.469969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:35.470011 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:35.470451 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:35.470455 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:35.472212 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
W20260924 21:17:35.472431 30121 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:35.467863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:45847 (local address 127.20.115.254:36839)
0924 21:17:35.467987 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:35.467988 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:35.468043 (+ 55us) server_negotiation.cc:407] Connection header received
0924 21:17:35.468080 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:35.468081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:35.468091 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:35.468150 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:35.468432 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.468991 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.469699 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.469900 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.469949 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:35.472307 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:45847: BlockingRecv error: recv got EOF from 127.20.115.193:45847 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":74,"threads_started":1}
Sep 24 21:17:35 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:35 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:35.621862 30122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:35.617186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:40527)
0924 21:17:35.617369 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:35.617381 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:35.617448 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:35.617602 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:35.617603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:35.617792 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:35.617839 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:35.617840 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.618533 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.618534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:35.619085 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:35.619086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.619451 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.619452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:35.619510 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:35.619972 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:35.619977 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:35.621791 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":96,"threads_started":1}
W20260924 21:17:35.621978 30121 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:35.617285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:40527 (local address 127.20.115.254:36839)
0924 21:17:35.617358 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:35.617360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:35.617428 (+ 68us) server_negotiation.cc:407] Connection header received
0924 21:17:35.617503 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:35.617504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:35.617518 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:35.617596 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:35.617921 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.618431 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.619173 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.619366 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.619451 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:35.621880 (+ 2429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:40527: BlockingRecv error: recv got EOF from 127.20.115.194:40527 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Sep 24 21:17:35 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:35 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:35.743839 30123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:35.739201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:59327)
0924 21:17:35.739320 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:35.739339 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:35.739390 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:35.739523 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:35.739524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:35.739687 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:35.739731 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:35.739732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.740421 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.740422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:35.741078 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:35.741080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.741425 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.741426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:35.741475 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:35.741927 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:35.741932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:35.743771 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":68,"threads_started":1}
W20260924 21:17:35.743934 30121 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:35.739284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:59327 (local address 127.20.115.254:36839)
0924 21:17:35.739382 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:35.739384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:35.739397 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:17:35.739432 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:35.739433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:35.739446 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:35.739508 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:35.739840 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.740337 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.741155 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:35.741337 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:35.741395 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:35.743864 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:59327: BlockingRecv error: recv got EOF from 127.20.115.195:59327 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 24 21:17:36 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:36 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:36.477375 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:36.472843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:36699)
0924 21:17:36.472973 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:36.472986 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:36.473049 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:36.473198 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:36.473199 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:36.473417 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:36.473471 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:36.473472 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.474180 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.474181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:36.474722 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:36.474724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.475100 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.475101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:36.475159 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:36.475593 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:36.475597 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:36.477320 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":72,"threads_started":1}
W20260924 21:17:36.477505 30125 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:36.472909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:36699 (local address 127.20.115.254:36839)
0924 21:17:36.473046 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:36.473053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:36.473061 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:36.473105 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:36.473106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:36.473120 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:36.473191 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:36.473566 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.474086 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.474818 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.475023 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.475102 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:36.477416 (+ 2314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:36699: BlockingRecv error: recv got EOF from 127.20.115.193:36699 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
Sep 24 21:17:36 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:36 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:36.627274 30126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:36.622470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:42871)
0924 21:17:36.622652 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:36.622664 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:36.622754 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:36.622918 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:36.622918 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:36.623160 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:36.623221 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:36.623224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.624118 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.624119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:36.624645 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:36.624647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.625019 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.625020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:36.625064 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:36.625489 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:36.625494 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:36.627216 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
W20260924 21:17:36.627384 30125 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:36.622565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:42871 (local address 127.20.115.254:36839)
0924 21:17:36.622635 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:36.622638 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:36.622733 (+ 95us) server_negotiation.cc:407] Connection header received
0924 21:17:36.622823 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:36.622824 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:36.622844 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:36.622910 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:36.623365 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.624042 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.624724 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.624955 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.625012 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:36.627309 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:42871: BlockingRecv error: recv got EOF from 127.20.115.194:42871 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 24 21:17:36 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:36 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:36.749151 30127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:36.744387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:38923)
0924 21:17:36.744590 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:36.744602 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:36.744664 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:36.744858 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:36.744859 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:36.745038 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:36.745083 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:36.745085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.745711 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.745712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:36.746228 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:36.746229 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.746588 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.746589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:36.746632 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:36.747158 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:36.747163 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:36.749090 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
W20260924 21:17:36.749265 30125 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:36.744470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:38923 (local address 127.20.115.254:36839)
0924 21:17:36.744551 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:36.744555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:36.744691 (+ 136us) server_negotiation.cc:407] Connection header received
0924 21:17:36.744757 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:36.744757 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:36.744795 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:36.744835 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:36.745148 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.745643 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.746317 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:36.746513 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:36.746574 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:36.749188 (+ 2614us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:38923: BlockingRecv error: recv got EOF from 127.20.115.195:38923 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 24 21:17:37 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:37 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:37.483045 30128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:37.477834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:53299)
0924 21:17:37.478013 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:37.478024 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:37.478073 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:37.478207 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:37.478207 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:37.478384 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:37.478437 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:37.478439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.479188 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.479189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:37.479755 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:37.479757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.480139 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.480139 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:37.480181 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:37.480749 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:37.480756 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:37.482932 (+ 2176us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":117,"threads_started":1}
W20260924 21:17:37.483089 30129 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:37.477948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:53299 (local address 127.20.115.254:36839)
0924 21:17:37.478080 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:37.478082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:37.478091 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:37.478114 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:37.478115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:37.478124 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:37.478189 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:37.478544 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.479092 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.479846 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.480068 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.480126 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:37.483012 (+ 2886us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:53299: BlockingRecv error: recv got EOF from 127.20.115.193:53299 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
Sep 24 21:17:37 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:37 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:37.635332 30130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:37.627863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:51457)
0924 21:17:37.628143 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:37.628156 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:37.628210 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:37.628424 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:37.628426 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:37.628629 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:37.628693 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:37.628710 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.629946 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.629947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:37.631126 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:37.631129 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.631843 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.631844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:37.631915 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:37.632398 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:37.632404 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:37.635214 (+ 2810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":84,"threads_started":1}
W20260924 21:17:37.635370 30129 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:37.628110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:51457 (local address 127.20.115.254:36839)
0924 21:17:37.628193 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:37.628195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:37.628206 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:17:37.628301 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:37.628302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:37.628336 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:37.628407 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:37.628986 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.629811 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.631225 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.631739 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.631842 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:37.635301 (+ 3459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:51457: BlockingRecv error: recv got EOF from 127.20.115.194:51457 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 24 21:17:37 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:37 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:37.754436 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:37.749682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:42019)
0924 21:17:37.749884 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:37.749897 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:37.749975 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:37.750130 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:37.750131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:37.750319 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:37.750362 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:37.750363 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.751083 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.751084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:37.751627 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:37.751629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.752000 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.752001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:37.752065 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:37.752508 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:37.752513 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:37.754366 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":113,"threads_started":1}
W20260924 21:17:37.754567 30129 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:37.749811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:42019 (local address 127.20.115.254:36839)
0924 21:17:37.749888 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:37.749891 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:37.749966 (+ 75us) server_negotiation.cc:407] Connection header received
0924 21:17:37.750035 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:37.750036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:37.750051 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:37.750124 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:37.750478 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.750985 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.751717 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:37.751906 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:37.752001 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:37.754485 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:42019: BlockingRecv error: recv got EOF from 127.20.115.195:42019 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Sep 24 21:17:38 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:38 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:38.488178 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:38.483640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:50169)
0924 21:17:38.483773 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:38.483783 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:38.483842 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:38.483989 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:38.483990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:38.484153 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:38.484200 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:38.484201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.484894 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.484894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:38.485428 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:38.485429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.485757 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.485757 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:38.485799 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:38.486294 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:38.486301 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:38.488051 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
W20260924 21:17:38.488193 30133 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:38.483656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:50169 (local address 127.20.115.254:36839)
0924 21:17:38.483789 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:38.483790 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:38.483826 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:17:38.483891 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:38.483892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:38.483902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:38.483978 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:38.484268 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.484761 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.485498 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.485689 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.485741 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:38.488140 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:50169: BlockingRecv error: recv got EOF from 127.20.115.193:50169 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":56,"threads_started":1}
Sep 24 21:17:38 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:38 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
W20260924 21:17:38.643836 30133 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:38.636027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:50611 (local address 127.20.115.254:36839)
0924 21:17:38.636073 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:38.636075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:38.636280 (+ 205us) server_negotiation.cc:407] Connection header received
0924 21:17:38.636344 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:38.636345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:38.636355 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:38.636408 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:38.636715 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.637226 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.637995 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.638202 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.638259 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:38.643775 (+ 5516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:50611: BlockingRecv error: recv got EOF from 127.20.115.194:50611 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:17:38.643807 30134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:38.635949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:50611)
0924 21:17:38.636218 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:38.636236 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:38.636290 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:38.636429 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:38.636430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:38.636603 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:38.636650 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:38.636652 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.637327 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.637328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:38.637904 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:38.637906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.638276 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.638277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:38.638318 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:38.641652 (+ 3334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:38.641657 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:38.643687 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":212,"threads_started":1}
Sep 24 21:17:38 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:38 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.195@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:38.759905 30135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:38.755003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.195:39713)
0924 21:17:38.755137 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:38.755151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:38.755238 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:38.755412 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:38.755413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:38.755590 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:38.755635 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:38.755637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.756421 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.756423 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:38.756994 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:38.756997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.757485 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.757486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:38.757551 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:38.758080 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:38.758085 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:38.759808 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
W20260924 21:17:38.759958 30133 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:38.755108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:39713 (local address 127.20.115.254:36839)
0924 21:17:38.755159 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:38.755161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:38.755217 (+ 56us) server_negotiation.cc:407] Connection header received
0924 21:17:38.755311 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:38.755312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:38.755327 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:38.755404 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:38.755713 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.756335 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.757082 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:38.757394 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:38.757482 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:38.759894 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.195:39713: BlockingRecv error: recv got EOF from 127.20.115.195:39713 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 24 21:17:39 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:39 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.193@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:39.493253 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:39.488798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.193:33253)
0924 21:17:39.488949 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:39.488962 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:39.489053 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:39.489213 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:39.489213 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:39.489367 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:39.489416 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:39.489417 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.490113 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.490114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:39.490656 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:39.490658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.490986 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.490987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:39.491031 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:39.491455 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:39.491460 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:39.493194 (+ 1734us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
W20260924 21:17:39.493336 30137 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:39.488859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:33253 (local address 127.20.115.254:36839)
0924 21:17:39.489005 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:39.489010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:39.489025 (+ 15us) server_negotiation.cc:407] Connection header received
0924 21:17:39.489121 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:39.489122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:39.489135 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:39.489210 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:39.489492 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.489989 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.490732 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.490921 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.490971 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:39.493281 (+ 2310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.193:33253: BlockingRecv error: recv got EOF from 127.20.115.193:33253 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
Sep 24 21:17:39 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
Sep 24 21:17:39 dist-test-slave-cxpd krb5kdc[29079](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.115.194@KRBTEST.COM for oryx/127.20.115.254@KRBTEST.COM, Server not found in Kerberos database
I20260924 21:17:39.648747 30138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:39.644420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36839 (local address 127.20.115.194:49163)
0924 21:17:39.644567 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:39.644585 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:39.644642 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:39.644819 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:39.644821 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:39.644987 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:39.645035 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:39.645036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.645706 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.645708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:39.646240 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:39.646241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.646557 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.646558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:39.646640 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:39.647055 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:39.647059 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:39.648692 (+ 1633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.115.254:36839: Server oryx/127.20.115.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
W20260924 21:17:39.648908 30137 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:39.644490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:49163 (local address 127.20.115.254:36839)
0924 21:17:39.644557 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:39.644559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:39.644643 (+ 84us) server_negotiation.cc:407] Connection header received
0924 21:17:39.644703 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:39.644703 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:39.644714 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:39.644755 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:39.645098 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.645638 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.646318 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.646495 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.646544 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:39.648822 (+ 2278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.115.194:49163: BlockingRecv error: recv got EOF from 127.20.115.194:49163 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:17:39.670583 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 29662
I20260924 21:17:39.675345 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 29814
I20260924 21:17:39.679513 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 29946
I20260924 21:17:39.683601 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 29592
2026-09-24T21:17:39Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17400 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-24T21:17:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:39Z Disabled control of system clock
I20260924 21:17:39.710718 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:38023
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:38023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260924 21:17:39.789496 30142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:39.789691 30142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:39.789710 30142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:39.790889 30142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:39.790930 30142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:39.790942 30142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:39.790956 30142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:39.791007 30142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:39.792320 30142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:38023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:38023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:39.792564 30142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:39.792881 30142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:39.797531 30150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:39.797616 30142 server_base.cc:1034] running on GCE node
W20260924 21:17:39.797525 30147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:39.797525 30148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:39.797955 30142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:39.798193 30142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:39.799328 30142 hybrid_clock.cc:648] HybridClock initialized: now 1790284659799306 us; error 48 us; skew 500 ppm
I20260924 21:17:39.800530 30142 webserver.cc:460] Webserver started at http://127.20.115.254:41781/ using document root <none> and password file <none>
I20260924 21:17:39.800706 30142 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:39.800743 30142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:39.800891 30142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:39.801887 30142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "2b1e7223d2eb4ace9e43bf063820a781"
format_stamp: "Formatted at 2026-09-24 21:17:39 on dist-test-slave-cxpd"
server_key: "403413d99e8b3afcd78d9e21b3df3f32"
server_key_iv: "1c7472eb3b60ff76148bd4daa1a59572"
server_key_version: "encryptionkey@0"
I20260924 21:17:39.802173 30142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "2b1e7223d2eb4ace9e43bf063820a781"
format_stamp: "Formatted at 2026-09-24 21:17:39 on dist-test-slave-cxpd"
server_key: "403413d99e8b3afcd78d9e21b3df3f32"
server_key_iv: "1c7472eb3b60ff76148bd4daa1a59572"
server_key_version: "encryptionkey@0"
I20260924 21:17:39.803347 30142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:39.803974 30156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:39.804155 30142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:39.804226 30142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "2b1e7223d2eb4ace9e43bf063820a781"
format_stamp: "Formatted at 2026-09-24 21:17:39 on dist-test-slave-cxpd"
server_key: "403413d99e8b3afcd78d9e21b3df3f32"
server_key_iv: "1c7472eb3b60ff76148bd4daa1a59572"
server_key_version: "encryptionkey@0"
I20260924 21:17:39.804318 30142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:39.811965 30142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:39.812230 30142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:39.812363 30142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:39.816236 30142 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:38023
I20260924 21:17:39.816300 30208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:38023 every 8 connection(s)
I20260924 21:17:39.816596 30142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:39.817219 30209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:39.819639 30209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781: Bootstrap starting.
I20260924 21:17:39.820211 30209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:39.820516 30209 log.cc:824] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:39.821197 30209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781: No bootstrap required, opened a new log
I20260924 21:17:39.822701 30209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1e7223d2eb4ace9e43bf063820a781" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38023 } }
I20260924 21:17:39.822841 30209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:39.822865 30209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b1e7223d2eb4ace9e43bf063820a781, State: Initialized, Role: FOLLOWER
I20260924 21:17:39.823027 30209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1e7223d2eb4ace9e43bf063820a781" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38023 } }
I20260924 21:17:39.823088 30209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:39.823112 30209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:39.823173 30209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:39.823671 30209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1e7223d2eb4ace9e43bf063820a781" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38023 } }
I20260924 21:17:39.823778 30209 leader_election.cc:302] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b1e7223d2eb4ace9e43bf063820a781; no voters:
I20260924 21:17:39.823956 30209 leader_election.cc:288] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:39.824000 30212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:39.824189 30212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [term 1 LEADER]: Becoming Leader. State: Replica: 2b1e7223d2eb4ace9e43bf063820a781, State: Running, Role: LEADER
I20260924 21:17:39.824196 30209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:39.824290 30212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1e7223d2eb4ace9e43bf063820a781" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38023 } }
I20260924 21:17:39.824602 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 30142
I20260924 21:17:39.824627 30213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b1e7223d2eb4ace9e43bf063820a781" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1e7223d2eb4ace9e43bf063820a781" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38023 } } }
I20260924 21:17:39.824707 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:39.824698 30213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:39.824743 30214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b1e7223d2eb4ace9e43bf063820a781. Latest consensus state: current_term: 1 leader_uuid: "2b1e7223d2eb4ace9e43bf063820a781" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b1e7223d2eb4ace9e43bf063820a781" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38023 } } }
I20260924 21:17:39.824862 30214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:39.824993 20943 external_mini_cluster.cc:1428] Setting key 6a1e39f3b4a110d6fda7b40b99f51518
I20260924 21:17:39.825058 30219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:39.825572 30219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:39.827237 30219 catalog_manager.cc:1347] Generated new cluster ID: 0ad98a1ecb4543bead39132c3d356490
I20260924 21:17:39.827292 30219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:39.830927 30229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:39.825613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55404 (local address 127.20.115.254:38023)
0924 21:17:39.826167 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:39.826174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:39.826221 (+ 47us) server_negotiation.cc:407] Connection header received
0924 21:17:39.826380 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:39.826387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:39.826452 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:39.826597 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:39.826800 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.827823 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.828863 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.829460 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.829582 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:39.830253 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:39.830257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:39.830257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:39.830469 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:39.830509 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:39.830524 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:39.830596 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:39.830660 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 21:17:39.830734 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":65,"threads_started":1}
I20260924 21:17:39.840921 30219 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:39.841503 30219 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:39.848198 30219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b1e7223d2eb4ace9e43bf063820a781: Generated new TSK 0
I20260924 21:17:39.848389 30219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:17:39.852391 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:38023
--builtin_ntp_servers=127.20.115.212:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260924 21:17:39.932224 30233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:39.932447 30233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:39.932480 30233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:39.933672 30233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:39.933732 30233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:39.933773 30233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:39.935128 30233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:38023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:39.935376 30233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:39.935667 30233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:39.941507 30238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:39.941536 30239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:39.941745 30241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:39.941756 30233 server_base.cc:1034] running on GCE node
I20260924 21:17:39.941984 30233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:39.942242 30233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:39.943408 30233 hybrid_clock.cc:648] HybridClock initialized: now 1790284659943385 us; error 39 us; skew 500 ppm
I20260924 21:17:39.944667 30233 webserver.cc:460] Webserver started at http://127.20.115.193:46435/ using document root <none> and password file <none>
I20260924 21:17:39.944947 30233 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:39.945016 30233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:39.945155 30233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:39.946178 30233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "87d9b78522c540b887ddbfe773f415df"
format_stamp: "Formatted at 2026-09-24 21:17:39 on dist-test-slave-cxpd"
server_key: "8f07c5fe1fe305dabe88c2e480f0398d"
server_key_iv: "2bdf0c2d541b0353872297f9cacee078"
server_key_version: "encryptionkey@0"
I20260924 21:17:39.946535 30233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "87d9b78522c540b887ddbfe773f415df"
format_stamp: "Formatted at 2026-09-24 21:17:39 on dist-test-slave-cxpd"
server_key: "8f07c5fe1fe305dabe88c2e480f0398d"
server_key_iv: "2bdf0c2d541b0353872297f9cacee078"
server_key_version: "encryptionkey@0"
I20260924 21:17:39.947847 30233 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:39.948602 30247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:39.948772 30233 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:39.948876 30233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "87d9b78522c540b887ddbfe773f415df"
format_stamp: "Formatted at 2026-09-24 21:17:39 on dist-test-slave-cxpd"
server_key: "8f07c5fe1fe305dabe88c2e480f0398d"
server_key_iv: "2bdf0c2d541b0353872297f9cacee078"
server_key_version: "encryptionkey@0"
I20260924 21:17:39.948949 30233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:39.965364 30233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:39.965649 30233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:39.965796 30233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:39.965998 30233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:39.966318 30233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:39.966352 30233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:39.966394 30233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:39.966419 30233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:39.972200 30233 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:42139
I20260924 21:17:39.972259 30360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:42139 every 8 connection(s)
I20260924 21:17:39.972587 30233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:17:39.977331 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 30233
I20260924 21:17:39.977435 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:39.977515 30229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:39.973301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:48715 (local address 127.20.115.254:38023)
0924 21:17:39.973367 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:39.973369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:39.973615 (+ 246us) server_negotiation.cc:407] Connection header received
0924 21:17:39.974063 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:39.974064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:39.974076 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:39.974128 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:39.974737 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.975227 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.976147 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.976321 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.976370 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:39.977202 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:39.977203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:39.977203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:39.977207 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:39.977219 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:39.977223 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:39.977267 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:39.977368 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 21:17:39.977413 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:17:39.977653 20943 external_mini_cluster.cc:1428] Setting key a52defd435c92ff094a2e8ceaada13a7
I20260924 21:17:39.977715 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:39.973275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:38023 (local address 127.20.115.193:48715)
0924 21:17:39.973493 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:39.973531 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:39.973985 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:39.974185 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:39.974188 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:39.974196 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:39.974669 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:39.974673 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.975306 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.975307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:39.976074 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:39.976076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:39.976399 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:39.976400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:39.976501 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:39.977047 (+ 546us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:39.977048 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:39.977073 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:39.977094 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:39.977282 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:39.977287 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:39.977290 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:39.977362 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:17:39.977491 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":121,"threads_started":1}
I20260924 21:17:39.978209 30361 heartbeater.cc:347] Connected to a master server at 127.20.115.254:38023
I20260924 21:17:39.978327 30361 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:39.978495 30361 heartbeater.cc:511] Master 127.20.115.254:38023 requested a full tablet report, sending...
I20260924 21:17:39.978614 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:38023
--builtin_ntp_servers=127.20.115.212:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 21:17:39.978988 30173 ts_manager.cc:194] Registered new tserver with Master: 87d9b78522c540b887ddbfe773f415df (127.20.115.193:42139)
I20260924 21:17:39.979602 30173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.193:48715
W20260924 21:17:40.058497 30364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:40.058681 30364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:40.058702 30364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:40.059820 30364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:40.059856 30364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:40.059880 30364 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:40.061246 30364 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:38023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:40.061479 30364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:40.061750 30364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:40.066545 30369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:40.066526 30370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.066526 30364 server_base.cc:1034] running on GCE node
W20260924 21:17:40.066519 30372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.066916 30364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:40.067183 30364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:40.068369 30364 hybrid_clock.cc:648] HybridClock initialized: now 1790284660068356 us; error 42 us; skew 500 ppm
I20260924 21:17:40.069639 30364 webserver.cc:460] Webserver started at http://127.20.115.194:37173/ using document root <none> and password file <none>
I20260924 21:17:40.069896 30364 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:40.069968 30364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:40.070116 30364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:40.071159 30364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "c10e3d91ab4649b88f483c1dbb5c811c"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "219706478096b230dca2eb1ada7cc27f"
server_key_iv: "aa9ca8f822437fbe7524b5b12c782656"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.071496 30364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "c10e3d91ab4649b88f483c1dbb5c811c"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "219706478096b230dca2eb1ada7cc27f"
server_key_iv: "aa9ca8f822437fbe7524b5b12c782656"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.072741 30364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:17:40.073519 30378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.073683 30364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:40.073781 30364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "c10e3d91ab4649b88f483c1dbb5c811c"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "219706478096b230dca2eb1ada7cc27f"
server_key_iv: "aa9ca8f822437fbe7524b5b12c782656"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.073899 30364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:40.092247 30364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:40.092564 30364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:40.092715 30364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:40.092970 30364 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:40.093294 30364 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:40.093351 30364 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.093386 30364 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:40.093415 30364 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.099187 30364 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:32791
I20260924 21:17:40.099236 30491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:32791 every 8 connection(s)
I20260924 21:17:40.099545 30364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:17:40.103601 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 30364
I20260924 21:17:40.103706 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:40.103910 20943 external_mini_cluster.cc:1428] Setting key 0bbd2c6daabc981af688c130f056e855
I20260924 21:17:40.104925 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:38023
--builtin_ntp_servers=127.20.115.212:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260924 21:17:40.104904 30229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.100248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:42101 (local address 127.20.115.254:38023)
0924 21:17:40.100316 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:40.100319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:40.100535 (+ 216us) server_negotiation.cc:407] Connection header received
0924 21:17:40.101135 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:40.101136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:40.101147 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:40.101210 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:40.101863 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:40.102347 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:40.103265 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:40.103453 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:40.103504 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:40.104628 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:40.104629 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:40.104630 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:40.104634 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:40.104649 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:40.104653 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:40.104703 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:40.104792 (+ 89us) server_negotiation.cc:299] Negotiation successful
0924 21:17:40.104828 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:17:40.105126 30494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.100208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:38023 (local address 127.20.115.194:42101)
0924 21:17:40.100420 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:40.100461 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:40.101018 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:40.101287 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:40.101291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:40.101299 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:40.101790 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:40.101794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:40.102424 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:40.102425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:40.103191 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:40.103193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:40.103635 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:40.103636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:40.103806 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:40.104483 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:40.104483 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:40.104510 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:40.104531 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:40.104703 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:40.104709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:40.104712 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:40.104794 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:17:40.104893 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":92,"threads_started":1}
I20260924 21:17:40.105610 30492 heartbeater.cc:347] Connected to a master server at 127.20.115.254:38023
I20260924 21:17:40.105711 30492 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:40.105902 30492 heartbeater.cc:511] Master 127.20.115.254:38023 requested a full tablet report, sending...
I20260924 21:17:40.106272 30173 ts_manager.cc:194] Registered new tserver with Master: c10e3d91ab4649b88f483c1dbb5c811c (127.20.115.194:32791)
I20260924 21:17:40.106769 30173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.194:42101
W20260924 21:17:40.185240 30495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:40.185437 30495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:40.185457 30495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:40.186535 30495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:40.186566 30495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:40.186590 30495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:40.187842 30495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:38023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:40.188022 30495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:40.188258 30495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:40.193380 30503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:40.193418 30500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.193470 30495 server_base.cc:1034] running on GCE node
W20260924 21:17:40.193423 30501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.193904 30495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:40.194195 30495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:40.195355 30495 hybrid_clock.cc:648] HybridClock initialized: now 1790284660195347 us; error 40 us; skew 500 ppm
I20260924 21:17:40.196633 30495 webserver.cc:460] Webserver started at http://127.20.115.195:33177/ using document root <none> and password file <none>
I20260924 21:17:40.196943 30495 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:40.197013 30495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:40.197153 30495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:40.198189 30495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "22bdbc5a80cf4ddf84394bc3e767078c"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "27030e8d9328657a3d3e7da593dbdce9"
server_key_iv: "8e12ca69206a43479e784fae16f51539"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.198567 30495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "22bdbc5a80cf4ddf84394bc3e767078c"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "27030e8d9328657a3d3e7da593dbdce9"
server_key_iv: "8e12ca69206a43479e784fae16f51539"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.199843 30495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:40.200623 30509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.200870 30495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:40.200976 30495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "22bdbc5a80cf4ddf84394bc3e767078c"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "27030e8d9328657a3d3e7da593dbdce9"
server_key_iv: "8e12ca69206a43479e784fae16f51539"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.201074 30495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:40.225605 30495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:40.225864 30495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:40.225960 30495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:40.226137 30495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:40.226468 30495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:40.226502 30495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.226555 30495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:40.226586 30495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.232276 30495 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:39597
I20260924 21:17:40.232322 30622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:39597 every 8 connection(s)
I20260924 21:17:40.232631 30495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:17:40.237673 30229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.233340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:40257 (local address 127.20.115.254:38023)
0924 21:17:40.233405 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:40.233407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:40.233635 (+ 228us) server_negotiation.cc:407] Connection header received
0924 21:17:40.234209 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:40.234211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:40.234222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:40.234284 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:40.234960 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:40.235461 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:40.236432 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:40.236606 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:40.236654 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:40.237436 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:40.237437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:40.237438 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:40.237442 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:40.237456 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:40.237460 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:40.237510 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:40.237592 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 21:17:40.237615 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:17:40.237896 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.233295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:38023 (local address 127.20.115.195:40257)
0924 21:17:40.233500 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:40.233531 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:40.234019 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:40.234358 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:40.234362 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:40.234370 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:40.234878 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:40.234882 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:40.235565 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:40.235566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:40.236354 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:40.236356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:40.236685 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:40.236686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:40.236815 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:40.237285 (+ 470us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:40.237285 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:40.237314 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:40.237335 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:40.237514 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:40.237517 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:40.237519 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:17:40.237566 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 21:17:40.237638 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":71,"threads_started":1}
I20260924 21:17:40.238387 30623 heartbeater.cc:347] Connected to a master server at 127.20.115.254:38023
I20260924 21:17:40.238490 30623 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:40.238658 30623 heartbeater.cc:511] Master 127.20.115.254:38023 requested a full tablet report, sending...
I20260924 21:17:40.239018 30173 ts_manager.cc:194] Registered new tserver with Master: 22bdbc5a80cf4ddf84394bc3e767078c (127.20.115.195:39597)
I20260924 21:17:40.239364 30173 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.195:40257
I20260924 21:17:40.240270 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 30495
I20260924 21:17:40.240339 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:40.240542 20943 external_mini_cluster.cc:1428] Setting key 0d2924a7b9024f501714578fb9f1f6c3
I20260924 21:17:40.241485 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260924 21:17:40.243351 30229 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:40.242865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55412 (local address 127.20.115.254:38023)
0924 21:17:40.242923 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:40.242925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:40.243054 (+ 129us) server_negotiation.cc:407] Connection header received
0924 21:17:40.243076 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:40.243077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:40.243088 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:40.243141 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:40.243282 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55412: BlockingRecv error: recv got EOF from 127.0.0.1:55412 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:17:40.243418 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 30233
I20260924 21:17:40.248584 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 30364
I20260924 21:17:40.252767 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 30495
I20260924 21:17:40.256968 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 30142
2026-09-24T21:17:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (572 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-24T21:17:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:40Z Disabled control of system clock
I20260924 21:17:40.283110 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:35933
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:35933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 21:17:40.361835 30634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:40.362030 30634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:40.362051 30634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:40.363497 30634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:40.363538 30634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:40.363557 30634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:40.363570 30634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:40.363582 30634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:40.364969 30634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:35933
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:35933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:40.365221 30634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:40.365499 30634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:40.370183 30642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:40.370183 30639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.370298 30634 server_base.cc:1034] running on GCE node
W20260924 21:17:40.370198 30640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.370652 30634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:40.371022 30634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:40.372190 30634 hybrid_clock.cc:648] HybridClock initialized: now 1790284660372184 us; error 44 us; skew 500 ppm
I20260924 21:17:40.373446 30634 webserver.cc:460] Webserver started at http://127.20.115.254:37061/ using document root <none> and password file <none>
I20260924 21:17:40.373675 30634 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:40.373723 30634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:40.373896 30634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:40.374930 30634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "c4effce8367f84f199c6cb394ce903b7"
server_key_iv: "da8ef422ec9645140d6b44b92106a5b0"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.375283 30634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "c4effce8367f84f199c6cb394ce903b7"
server_key_iv: "da8ef422ec9645140d6b44b92106a5b0"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.376541 30634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:40.377326 30648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.377506 30634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.377606 30634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "c4effce8367f84f199c6cb394ce903b7"
server_key_iv: "da8ef422ec9645140d6b44b92106a5b0"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.377691 30634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:40.380890 30634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:40.381142 30634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:40.381289 30634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:40.384981 30634 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:35933
I20260924 21:17:40.385011 30700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:35933 every 8 connection(s)
I20260924 21:17:40.385324 30634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:40.385911 30701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:40.387022 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 30634
I20260924 21:17:40.387136 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:40.387331 20943 external_mini_cluster.cc:1428] Setting key eec5d6c21c55aedbb3ece11366c3299d
I20260924 21:17:40.389655 30701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4: Bootstrap starting.
I20260924 21:17:40.389884 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.387842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60410 (local address 127.20.115.254:35933)
0924 21:17:40.388041 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:40.388050 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:40.388079 (+ 29us) server_negotiation.cc:407] Connection header received
0924 21:17:40.388596 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:40.388605 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:40.388660 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:40.388828 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:40.389480 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:40.389484 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:40.389484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:40.389488 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:40.389533 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:40.389547 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:40.389627 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:40.389663 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 21:17:40.389715 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
I20260924 21:17:40.390420 30701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:40.390695 30701 log.cc:824] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:40.391372 30701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4: No bootstrap required, opened a new log
I20260924 21:17:40.392686 30701 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 35933 } }
I20260924 21:17:40.392879 30701 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:40.392933 30701 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0d4ca81a9b34abbb509cd0a1f8f75c4, State: Initialized, Role: FOLLOWER
I20260924 21:17:40.393054 30701 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 35933 } }
I20260924 21:17:40.393112 30701 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:40.393152 30701 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:40.393198 30701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:40.393685 30701 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 35933 } }
I20260924 21:17:40.393805 30701 leader_election.cc:302] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0d4ca81a9b34abbb509cd0a1f8f75c4; no voters:
I20260924 21:17:40.393987 30701 leader_election.cc:288] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:40.394230 30701 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:40.394275 30706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:40.394423 30706 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [term 1 LEADER]: Becoming Leader. State: Replica: b0d4ca81a9b34abbb509cd0a1f8f75c4, State: Running, Role: LEADER
I20260924 21:17:40.394590 30706 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 35933 } }
I20260924 21:17:40.395121 30708 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0d4ca81a9b34abbb509cd0a1f8f75c4. Latest consensus state: current_term: 1 leader_uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 35933 } } }
I20260924 21:17:40.395231 30708 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:40.395264 30707 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0d4ca81a9b34abbb509cd0a1f8f75c4" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 35933 } } }
I20260924 21:17:40.395318 30707 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:40.395474 30718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:17:40.395983 30718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:40.397487 30718 catalog_manager.cc:1347] Generated new cluster ID: 906f79f0afc746179cf9896067e2d3d0
I20260924 21:17:40.397547 30718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:40.408589 30718 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:40.408761 30718 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:40.425109 30718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0d4ca81a9b34abbb509cd0a1f8f75c4: Generated new TSK 0
I20260924 21:17:40.425330 30718 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:17:40.435158 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:35933
--builtin_ntp_servers=127.20.115.212:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 21:17:40.514642 30725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:40.514870 30725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:40.514909 30725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:40.516079 30725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:40.516141 30725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:40.516187 30725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:40.517591 30725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:35933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:40.517845 30725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:40.518129 30725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:40.522964 30730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.523146 30725 server_base.cc:1034] running on GCE node
W20260924 21:17:40.522964 30731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:40.522987 30733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.523442 30725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:40.523639 30725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:40.524819 30725 hybrid_clock.cc:648] HybridClock initialized: now 1790284660524807 us; error 26 us; skew 500 ppm
I20260924 21:17:40.526010 30725 webserver.cc:460] Webserver started at http://127.20.115.193:35725/ using document root <none> and password file <none>
I20260924 21:17:40.526258 30725 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:40.526326 30725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:40.526467 30725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:40.527508 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "d14f7998e7f745709f01be82336288e3"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "db381868a0057e5cb035e0c96622aee6"
server_key_iv: "8cd69409bf2f3c221032f775fe093344"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.527851 30725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "d14f7998e7f745709f01be82336288e3"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "db381868a0057e5cb035e0c96622aee6"
server_key_iv: "8cd69409bf2f3c221032f775fe093344"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.529111 30725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:40.529804 30739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.529970 30725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:40.530072 30725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "d14f7998e7f745709f01be82336288e3"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "db381868a0057e5cb035e0c96622aee6"
server_key_iv: "8cd69409bf2f3c221032f775fe093344"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.530153 30725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:40.533510 30725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:40.533751 30725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:40.533893 30725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:40.534093 30725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:40.534386 30725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:40.534442 30725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.534482 30725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:40.534508 30725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.540192 30725 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:35639
I20260924 21:17:40.540227 30852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:35639 every 8 connection(s)
I20260924 21:17:40.540529 30725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:17:40.543412 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.541254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:37363 (local address 127.20.115.254:35933)
0924 21:17:40.541315 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:40.541336 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:40.541539 (+ 203us) server_negotiation.cc:407] Connection header received
0924 21:17:40.542032 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:40.542034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:40.542041 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:40.542092 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:40.543178 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:40.543180 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:40.543180 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:40.543185 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:40.543198 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:40.543202 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:40.543249 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:40.543331 (+ 82us) server_negotiation.cc:299] Negotiation successful
0924 21:17:40.543345 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:17:40.543605 30855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.541221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:35933 (local address 127.20.115.193:37363)
0924 21:17:40.541425 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:40.541458 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:40.541911 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:40.542157 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:40.542160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:40.542170 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:40.543031 (+ 861us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:40.543032 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:40.543058 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:40.543084 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:40.543258 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:40.543262 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:40.543264 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:17:40.543340 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:17:40.543397 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260924 21:17:40.544113 30853 heartbeater.cc:347] Connected to a master server at 127.20.115.254:35933
I20260924 21:17:40.544202 30853 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:40.544309 30853 heartbeater.cc:511] Master 127.20.115.254:35933 requested a full tablet report, sending...
I20260924 21:17:40.544773 30665 ts_manager.cc:194] Registered new tserver with Master: d14f7998e7f745709f01be82336288e3 (127.20.115.193:35639)
I20260924 21:17:40.549876 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 30725
I20260924 21:17:40.549960 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:40.550127 20943 external_mini_cluster.cc:1428] Setting key f11232428a2f54769a1fcae34c0884cc
I20260924 21:17:40.551123 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:35933
--builtin_ntp_servers=127.20.115.212:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 21:17:40.633069 30856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:40.633281 30856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:40.633304 30856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:40.634577 30856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:40.634620 30856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:40.634645 30856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:40.635972 30856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:35933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:40.636176 30856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:40.636425 30856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:17:40.641429 30856 server_base.cc:1034] running on GCE node
W20260924 21:17:40.641361 30861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:40.641393 30864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:40.641360 30862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.641836 30856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:40.642105 30856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:40.643281 30856 hybrid_clock.cc:648] HybridClock initialized: now 1790284660643264 us; error 24 us; skew 500 ppm
I20260924 21:17:40.644548 30856 webserver.cc:460] Webserver started at http://127.20.115.194:46223/ using document root <none> and password file <none>
I20260924 21:17:40.644841 30856 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:40.644919 30856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:40.645066 30856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:40.646152 30856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "94b7a73a7cf94050be24e7e3b0b1c990"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "fdf9071be261ad5c6e093f8074ab506f"
server_key_iv: "8c73f5ff30c45bc899b3a03972d4c3a4"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.646507 30856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "94b7a73a7cf94050be24e7e3b0b1c990"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "fdf9071be261ad5c6e093f8074ab506f"
server_key_iv: "8c73f5ff30c45bc899b3a03972d4c3a4"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.647781 30856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:40.648512 30870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.648684 30856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:40.648782 30856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "94b7a73a7cf94050be24e7e3b0b1c990"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "fdf9071be261ad5c6e093f8074ab506f"
server_key_iv: "8c73f5ff30c45bc899b3a03972d4c3a4"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.648890 30856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:40.652081 30856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:40.652330 30856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:40.652472 30856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:40.652676 30856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:40.653007 30856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:40.653071 30856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.653112 30856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:40.653138 30856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.658732 30856 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:46535
I20260924 21:17:40.658774 30983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:46535 every 8 connection(s)
I20260924 21:17:40.659070 30856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:17:40.661886 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.659696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:41231 (local address 127.20.115.254:35933)
0924 21:17:40.659739 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:40.659741 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:40.659954 (+ 213us) server_negotiation.cc:407] Connection header received
0924 21:17:40.660438 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:40.660439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:40.660446 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:40.660505 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:40.661641 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:40.661643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:40.661643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:40.661648 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:40.661661 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:40.661666 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:40.661720 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:40.661799 (+ 79us) server_negotiation.cc:299] Negotiation successful
0924 21:17:40.661814 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:17:40.662068 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.659672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:35933 (local address 127.20.115.194:41231)
0924 21:17:40.659831 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:40.659862 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:40.660329 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:40.660578 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:40.660582 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:40.660591 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:40.661485 (+ 894us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:40.661486 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:40.661513 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:40.661547 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:40.661723 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:40.661728 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:40.661730 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:17:40.661806 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:17:40.661866 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":63,"threads_started":1}
I20260924 21:17:40.662514 30984 heartbeater.cc:347] Connected to a master server at 127.20.115.254:35933
I20260924 21:17:40.662603 30984 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:40.662711 30984 heartbeater.cc:511] Master 127.20.115.254:35933 requested a full tablet report, sending...
I20260924 21:17:40.663029 30665 ts_manager.cc:194] Registered new tserver with Master: 94b7a73a7cf94050be24e7e3b0b1c990 (127.20.115.194:46535)
I20260924 21:17:40.665575 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 30856
I20260924 21:17:40.665668 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:40.665928 20943 external_mini_cluster.cc:1428] Setting key d7d32d31c84b8776442315aa5e817a45
I20260924 21:17:40.666891 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:35933
--builtin_ntp_servers=127.20.115.212:45377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260924 21:17:40.760658 30987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:40.761053 30987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:40.761098 30987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:40.763414 30987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:40.763469 30987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:40.763520 30987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:40.766268 30987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:35933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:40.766532 30987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:40.766898 30987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:40.772441 30993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:40.772497 30995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:40.772425 30992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:40.772605 30987 server_base.cc:1034] running on GCE node
I20260924 21:17:40.772956 30987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:40.773244 30987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:40.774438 30987 hybrid_clock.cc:648] HybridClock initialized: now 1790284660774420 us; error 36 us; skew 500 ppm
I20260924 21:17:40.775987 30987 webserver.cc:460] Webserver started at http://127.20.115.195:37595/ using document root <none> and password file <none>
I20260924 21:17:40.776273 30987 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:40.776345 30987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:40.776495 30987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:40.778108 30987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "6c7c19e28cb74eada0fe9aa222495148"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "167256351a7fbe6520ce75c191d69686"
server_key_iv: "9bad0de64e91f51a3001a074ecefb65f"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.778563 30987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "6c7c19e28cb74eada0fe9aa222495148"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "167256351a7fbe6520ce75c191d69686"
server_key_iv: "9bad0de64e91f51a3001a074ecefb65f"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.780745 30987 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260924 21:17:40.781785 31001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.781984 30987 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:40.782100 30987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "6c7c19e28cb74eada0fe9aa222495148"
format_stamp: "Formatted at 2026-09-24 21:17:40 on dist-test-slave-cxpd"
server_key: "167256351a7fbe6520ce75c191d69686"
server_key_iv: "9bad0de64e91f51a3001a074ecefb65f"
server_key_version: "encryptionkey@0"
I20260924 21:17:40.782178 30987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:40.785404 30987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:40.785666 30987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:40.785809 30987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:40.786022 30987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:40.786320 30987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:40.786373 30987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.786404 30987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:40.786432 30987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:40.791817 30987 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:45577
I20260924 21:17:40.791913 31114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:45577 every 8 connection(s)
I20260924 21:17:40.792217 30987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:17:40.795837 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.792988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:42097 (local address 127.20.115.254:35933)
0924 21:17:40.793038 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:40.793041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:40.793337 (+ 296us) server_negotiation.cc:407] Connection header received
0924 21:17:40.794085 (+ 748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:40.794086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:40.794092 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:40.794171 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:40.795575 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:40.795576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:40.795576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:17:40.795580 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:40.795609 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:17:40.795615 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:40.795673 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:40.795774 (+ 101us) server_negotiation.cc:299] Negotiation successful
0924 21:17:40.795785 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:17:40.796100 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:40.792989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:35933 (local address 127.20.115.195:42097)
0924 21:17:40.793188 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:40.793233 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:40.793967 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:40.794245 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:40.794251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:40.794260 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:40.795410 (+ 1150us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:17:40.795411 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:40.795455 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:17:40.795494 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:40.795686 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:40.795692 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:40.795695 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:40.795787 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 21:17:40.795860 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260924 21:17:40.796656 31115 heartbeater.cc:347] Connected to a master server at 127.20.115.254:35933
I20260924 21:17:40.796749 31115 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:40.796945 31115 heartbeater.cc:511] Master 127.20.115.254:35933 requested a full tablet report, sending...
I20260924 21:17:40.797318 30665 ts_manager.cc:194] Registered new tserver with Master: 6c7c19e28cb74eada0fe9aa222495148 (127.20.115.195:45577)
I20260924 21:17:40.801803 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 30987
I20260924 21:17:40.801923 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:40.802126 20943 external_mini_cluster.cc:1428] Setting key 3c587c1f3055944f0ae45febbbfcbcac
I20260924 21:17:40.803045 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:17:40.805110 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 30725
W20260924 21:17:40.805151 30704 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:17:40.804430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60420 (local address 127.20.115.254:35933)
0924 21:17:40.804491 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:40.804493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:40.804590 (+ 97us) server_negotiation.cc:407] Connection header received
0924 21:17:40.804658 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:40.804660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:40.804666 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:40.804738 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:40.805070 (+ 332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60420: BlockingRecv error: recv got EOF from 127.0.0.1:60420 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:17:40.809849 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 30856
I20260924 21:17:40.814538 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 30987
I20260924 21:17:40.819947 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 30634
2026-09-24T21:17:40Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (563 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:40 dist-test-slave-cxpd krb5kdc[31125](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:40 dist-test-slave-cxpd krb5kdc[31125](info): set up 2 sockets
Sep 24 21:17:40 dist-test-slave-cxpd krb5kdc[31125](info): commencing operation
krb5kdc: starting...
W20260924 21:17:41.865701 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:41 dist-test-slave-cxpd krb5kdc[31125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:17:41.984185 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:44567
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:44567
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:42.064344 31138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:42.064597 31138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:42.064635 31138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:42.065828 31138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:17:42.065903 31138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:42.065927 31138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:42.065948 31138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:17:42.065979 31138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:17:42.067451 31138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:44567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:44567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:42.067731 31138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:42.067955 31138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:42.072800 31144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:42.072816 31143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:42.072816 31146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:42.072978 31138 server_base.cc:1034] running on GCE node
I20260924 21:17:42.073244 31138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:42.073554 31138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:42.074718 31138 hybrid_clock.cc:648] HybridClock initialized: now 1790284662074703 us; error 30 us; skew 500 ppm
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:42.077231 31138 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:17:42.077706 31138 webserver.cc:460] Webserver started at http://127.20.115.254:34343/ using document root <none> and password file <none>
I20260924 21:17:42.077937 31138 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:42.078006 31138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:42.078157 31138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:42.079182 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "98c052d533f44dfaba3976eb9f9153b0"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "7fb3a79508db2c4d0e888262d4e6d053"
server_key_iv: "8b4b496a05e905891f3d5790008b6051"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.079475 31138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "98c052d533f44dfaba3976eb9f9153b0"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "7fb3a79508db2c4d0e888262d4e6d053"
server_key_iv: "8b4b496a05e905891f3d5790008b6051"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.080696 31138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:42.081470 31153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.081670 31138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:17:42.081756 31138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "98c052d533f44dfaba3976eb9f9153b0"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "7fb3a79508db2c4d0e888262d4e6d053"
server_key_iv: "8b4b496a05e905891f3d5790008b6051"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.081812 31138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:42.098878 31138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:42.101161 31138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:42.101279 31138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:42.104914 31138 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:44567
I20260924 21:17:42.104995 31205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:44567 every 8 connection(s)
I20260924 21:17:42.105314 31138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:17:42.106371 31206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:42.108656 31206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0: Bootstrap starting.
I20260924 21:17:42.108750 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 31138
I20260924 21:17:42.108915 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:17:42.109133 20943 external_mini_cluster.cc:1428] Setting key 55998dbf22f1066724a2a848feccfa79
I20260924 21:17:42.109388 31206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:42.109757 31206 log.cc:824] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:42.110453 31206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0: No bootstrap required, opened a new log
I20260924 21:17:42.112239 31206 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c052d533f44dfaba3976eb9f9153b0" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44567 } }
I20260924 21:17:42.112406 31206 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:42.112455 31206 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98c052d533f44dfaba3976eb9f9153b0, State: Initialized, Role: FOLLOWER
I20260924 21:17:42.112609 31206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c052d533f44dfaba3976eb9f9153b0" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44567 } }
I20260924 21:17:42.112713 31206 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:17:42.112758 31206 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:17:42.112833 31206 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:42.113363 31206 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c052d533f44dfaba3976eb9f9153b0" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44567 } }
I20260924 21:17:42.113484 31206 leader_election.cc:302] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98c052d533f44dfaba3976eb9f9153b0; no voters:
I20260924 21:17:42.113677 31206 leader_election.cc:288] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:17:42.113699 31211 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:42.113834 31211 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [term 1 LEADER]: Becoming Leader. State: Replica: 98c052d533f44dfaba3976eb9f9153b0, State: Running, Role: LEADER
I20260924 21:17:42.113914 31211 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c052d533f44dfaba3976eb9f9153b0" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44567 } }
I20260924 21:17:42.113943 31206 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:17:42.114240 31211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98c052d533f44dfaba3976eb9f9153b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c052d533f44dfaba3976eb9f9153b0" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44567 } } }
I20260924 21:17:42.114312 31211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [sys.catalog]: This master's current role is: LEADER
I20260924 21:17:42.114445 31211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98c052d533f44dfaba3976eb9f9153b0. Latest consensus state: current_term: 1 leader_uuid: "98c052d533f44dfaba3976eb9f9153b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98c052d533f44dfaba3976eb9f9153b0" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44567 } } }
I20260924 21:17:42.114516 31211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0 [sys.catalog]: This master's current role is: LEADER
W20260924 21:17:42.115695 31227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:17:42.115764 31227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:17:42.115826 31216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:42.116012 31216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:17:42.117589 31216 catalog_manager.cc:1347] Generated new cluster ID: 5b142dc0c4424b898948c1c288875343
I20260924 21:17:42.117641 31216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:17:42.119966 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.109657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.20.115.254:44567)
0924 21:17:42.109873 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.109884 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.109915 (+ 31us) server_negotiation.cc:407] Connection header received
0924 21:17:42.110346 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.110353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.110399 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.110560 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:42.111181 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.112055 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.112999 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.113408 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.113559 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.116449 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:42.116453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:42.116453 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:42.116465 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:42.118524 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.118887 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.118890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.118890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.118989 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.119175 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.119176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.119176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.119452 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:42.119576 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.119657 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.119780 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":95,"threads_started":1}
I20260924 21:17:42.124614 31216 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:17:42.125155 31216 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:17:42.128424 31216 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0: Generated new TSK 0
I20260924 21:17:42.128603 31216 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:17:42.159426 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:44567
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:42.239949 31232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:42.240142 31232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:42.240165 31232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:42.241381 31232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:42.241422 31232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:42.241449 31232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:17:42.242875 31232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:44567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:42.243108 31232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:42.243335 31232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:42.248240 31238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:42.248286 31240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:42.248286 31237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:42.248430 31232 server_base.cc:1034] running on GCE node
I20260924 21:17:42.248711 31232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:42.248986 31232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:42.250152 31232 hybrid_clock.cc:648] HybridClock initialized: now 1790284662250125 us; error 42 us; skew 500 ppm
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:42.252470 31232 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:17:42.252947 31232 webserver.cc:460] Webserver started at http://127.20.115.193:33305/ using document root <none> and password file <none>
I20260924 21:17:42.253170 31232 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:42.253237 31232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:42.253391 31232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:42.254415 31232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "36260c417e3749aea1d75096eaad61c5"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "74bdf72b63ce31be2fbd65987b9c2d8c"
server_key_iv: "879c1dca81ae6631f87b4eb446ed3f08"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.254771 31232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "36260c417e3749aea1d75096eaad61c5"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "74bdf72b63ce31be2fbd65987b9c2d8c"
server_key_iv: "879c1dca81ae6631f87b4eb446ed3f08"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.256062 31232 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:17:42.256772 31247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.257036 31232 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:42.257133 31232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "36260c417e3749aea1d75096eaad61c5"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "74bdf72b63ce31be2fbd65987b9c2d8c"
server_key_iv: "879c1dca81ae6631f87b4eb446ed3f08"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.257218 31232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:42.267388 31232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:42.269162 31232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:42.269315 31232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:42.269526 31232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:42.269811 31232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:42.269866 31232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.269904 31232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:42.269932 31232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.275604 31232 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:45455
I20260924 21:17:42.275691 31360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:45455 every 8 connection(s)
I20260924 21:17:42.275970 31232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:42.284085 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 31232
I20260924 21:17:42.284206 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:17:42.284461 20943 external_mini_cluster.cc:1428] Setting key 5e97dd0149e41b9405974fb251b607a6
I20260924 21:17:42.285622 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.276673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:35623 (local address 127.20.115.254:44567)
0924 21:17:42.276717 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.276720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.276927 (+ 207us) server_negotiation.cc:407] Connection header received
0924 21:17:42.277454 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.277456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.277469 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.277532 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:42.278482 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.278989 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.279985 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.280217 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.280307 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.282491 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:42.282492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:42.282493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:42.282501 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:42.283725 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.284374 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.284375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.284376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.284449 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.284865 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.284866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.284866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.285007 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:42.285062 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.285416 (+ 354us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.285510 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:17:42.285815 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.276634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:44567 (local address 127.20.115.193:35623)
0924 21:17:42.276767 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:42.276833 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:42.277346 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:42.277607 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:42.277611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:42.278206 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:42.278381 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.278386 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.279105 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.279106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.279890 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.279891 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.280317 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.280319 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.280442 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.280971 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:42.280979 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:42.282382 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:42.283846 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:42.283853 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:42.283855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:42.284263 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:42.284528 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:42.284529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:42.284529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:42.284672 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:42.285089 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:42.285095 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:42.285260 (+ 165us) client_negotiation.cc:769] Sending connection context
0924 21:17:42.285395 (+ 135us) client_negotiation.cc:240] Negotiation successful
0924 21:17:42.285557 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260924 21:17:42.286283 31361 heartbeater.cc:347] Connected to a master server at 127.20.115.254:44567
I20260924 21:17:42.286365 31361 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:42.286571 31361 heartbeater.cc:511] Master 127.20.115.254:44567 requested a full tablet report, sending...
I20260924 21:17:42.287174 31170 ts_manager.cc:194] Registered new tserver with Master: 36260c417e3749aea1d75096eaad61c5 (127.20.115.193:45455)
I20260924 21:17:42.287809 31170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:35623
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:17:42.309674 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:44567
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:42.389814 31366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:42.390018 31366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:42.390041 31366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:42.391212 31366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:42.391258 31366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:42.391285 31366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:17:42.392755 31366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:44567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:42.393023 31366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:42.393265 31366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:42.398478 31372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:42.398483 31374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:42.398589 31366 server_base.cc:1034] running on GCE node
W20260924 21:17:42.398504 31371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:42.398929 31366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:42.399194 31366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:42.400360 31366 hybrid_clock.cc:648] HybridClock initialized: now 1790284662400338 us; error 38 us; skew 500 ppm
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:42.403088 31366 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:17:42.403577 31366 webserver.cc:460] Webserver started at http://127.20.115.194:44425/ using document root <none> and password file <none>
I20260924 21:17:42.403857 31366 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:42.403929 31366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:42.404080 31366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:42.405220 31366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "3a58871ffa6a4054aee58e16b67589f9"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "019f40567d7f1ed51014c61d9cce4876"
server_key_iv: "32cb157ed64b45f5a3665579ab04a17d"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.405612 31366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "3a58871ffa6a4054aee58e16b67589f9"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "019f40567d7f1ed51014c61d9cce4876"
server_key_iv: "32cb157ed64b45f5a3665579ab04a17d"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.407012 31366 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:17:42.407807 31381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.407989 31366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:17:42.408095 31366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "3a58871ffa6a4054aee58e16b67589f9"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "019f40567d7f1ed51014c61d9cce4876"
server_key_iv: "32cb157ed64b45f5a3665579ab04a17d"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.408169 31366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:42.419364 31366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:42.421322 31366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:42.421485 31366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:42.421696 31366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:42.422014 31366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:42.422048 31366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.422071 31366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:42.422116 31366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.427774 31366 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:33807
I20260924 21:17:42.427855 31494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:33807 every 8 connection(s)
I20260924 21:17:42.428164 31366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:42.434092 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 31366
I20260924 21:17:42.434177 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:17:42.434438 20943 external_mini_cluster.cc:1428] Setting key 2bb56a7c575534ff3a3eec37b6e4625c
I20260924 21:17:42.437379 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.428900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:57503 (local address 127.20.115.254:44567)
0924 21:17:42.428958 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.428960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.429148 (+ 188us) server_negotiation.cc:407] Connection header received
0924 21:17:42.429628 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.429629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.429638 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.429685 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:42.430585 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.431064 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.431979 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.432155 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.432204 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.434287 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:42.434289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:42.434289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:42.434296 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:42.435967 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.436348 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.436349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.436349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.436413 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.436691 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.436692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.436693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.436867 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:42.436942 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.437212 (+ 270us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.437288 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:17:42.437597 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.428897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:44567 (local address 127.20.115.194:57503)
0924 21:17:42.429048 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:42.429083 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:42.429544 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:42.429745 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:42.429748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:42.430320 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:42.430509 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.430514 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.431141 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.431142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.431909 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.431911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.432225 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.432226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.432322 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.432747 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:42.432755 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:42.434163 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:42.436048 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:42.436053 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:42.436055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:42.436269 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:42.436477 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:42.436477 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:42.436477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:42.436619 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:42.436933 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:42.436939 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:42.437079 (+ 140us) client_negotiation.cc:769] Sending connection context
0924 21:17:42.437191 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 21:17:42.437342 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":88,"threads_started":1}
I20260924 21:17:42.438035 31495 heartbeater.cc:347] Connected to a master server at 127.20.115.254:44567
I20260924 21:17:42.438150 31495 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:42.438323 31495 heartbeater.cc:511] Master 127.20.115.254:44567 requested a full tablet report, sending...
I20260924 21:17:42.438727 31170 ts_manager.cc:194] Registered new tserver with Master: 3a58871ffa6a4054aee58e16b67589f9 (127.20.115.194:33807)
I20260924 21:17:42.439252 31170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:57503
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:17:42.459302 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:44567
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:36679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:17:42.537984 31500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:17:42.538179 31500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:17:42.538201 31500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:17:42.539345 31500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:17:42.539386 31500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:17:42.539444 31500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:17:42.540966 31500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:44567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:17:42.541204 31500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:17:42.541427 31500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:17:42.546424 31506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:17:42.546416 31508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:42.546521 31500 server_base.cc:1034] running on GCE node
W20260924 21:17:42.546416 31505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:17:42.546834 31500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:17:42.547119 31500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:17:42.548300 31500 hybrid_clock.cc:648] HybridClock initialized: now 1790284662548290 us; error 26 us; skew 500 ppm
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31125](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:17:42.550695 31500 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:17:42.551106 31500 webserver.cc:460] Webserver started at http://127.20.115.195:32813/ using document root <none> and password file <none>
I20260924 21:17:42.551331 31500 fs_manager.cc:359] Metadata directory not provided
I20260924 21:17:42.551400 31500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:17:42.551564 31500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:17:42.552564 31500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "791c490f87b95c6da0b87821a35cb120"
server_key_iv: "23fa1fb0496d56ff4ddd84d179d7a4f9"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.552956 31500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "791c490f87b95c6da0b87821a35cb120"
server_key_iv: "23fa1fb0496d56ff4ddd84d179d7a4f9"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.554288 31500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:17:42.554979 31515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.555140 31500 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:17:42.555240 31500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed"
format_stamp: "Formatted at 2026-09-24 21:17:42 on dist-test-slave-cxpd"
server_key: "791c490f87b95c6da0b87821a35cb120"
server_key_iv: "23fa1fb0496d56ff4ddd84d179d7a4f9"
server_key_version: "encryptionkey@0"
I20260924 21:17:42.555312 31500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:17:42.567159 31500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:17:42.569099 31500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:17:42.569259 31500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:17:42.569482 31500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:17:42.569787 31500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:17:42.569842 31500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.569883 31500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:17:42.569912 31500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:17:42.575745 31500 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:44059
I20260924 21:17:42.575822 31628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:44059 every 8 connection(s)
I20260924 21:17:42.576149 31500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31125](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284662, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:17:42.583890 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 31500
I20260924 21:17:42.584002 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:17:42.584256 20943 external_mini_cluster.cc:1428] Setting key 53366325ad9376478a92520b89769b0a
I20260924 21:17:42.585299 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.576889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:36141 (local address 127.20.115.254:44567)
0924 21:17:42.576937 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.576939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.577156 (+ 217us) server_negotiation.cc:407] Connection header received
0924 21:17:42.577676 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.577677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.577691 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.577744 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:42.578655 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.579165 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.580154 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.580366 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.580451 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.582520 (+ 2069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:42.582521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:42.582521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:42.582529 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:42.583678 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.584219 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.584220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.584221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.584281 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.584557 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.584559 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.584559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.584695 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:42.584775 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.585129 (+ 354us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.585207 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:17:42.585500 31631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.576873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:44567 (local address 127.20.115.195:36141)
0924 21:17:42.577032 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:42.577066 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:42.577568 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:42.577814 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:42.577818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:42.578367 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:17:42.578548 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.578552 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.579258 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.579260 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.580053 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.580055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.580462 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.580463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.580588 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.581107 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:17:42.581114 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:17:42.582420 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:17:42.583783 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:42.583805 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:42.583807 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:42.584148 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:42.584360 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:17:42.584361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:17:42.584361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:17:42.584470 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:17:42.584766 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:17:42.584772 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:17:42.584966 (+ 194us) client_negotiation.cc:769] Sending connection context
0924 21:17:42.585105 (+ 139us) client_negotiation.cc:240] Negotiation successful
0924 21:17:42.585262 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260924 21:17:42.585922 31629 heartbeater.cc:347] Connected to a master server at 127.20.115.254:44567
I20260924 21:17:42.586021 31629 heartbeater.cc:464] Registering TS with master...
I20260924 21:17:42.586186 31629 heartbeater.cc:511] Master 127.20.115.254:44567 requested a full tablet report, sending...
I20260924 21:17:42.586529 31170 ts_manager.cc:194] Registered new tserver with Master: cfdfa9eee7184ba4b30a5a6b4ea86fed (127.20.115.195:44059)
I20260924 21:17:42.586997 31170 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:36141
I20260924 21:17:42.595868 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:17:42.603883 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.597381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56960 (local address 127.20.115.254:44567)
0924 21:17:42.597454 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.597456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.597464 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:17:42.597515 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.597516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.597525 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.597603 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:17:42.598237 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.599010 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.600160 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.600336 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.600393 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.601549 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:17:42.601551 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:17:42.601551 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:17:42.601557 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:17:42.602977 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.603297 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.603298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.603298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.603346 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:17:42.603477 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:17:42.603478 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:17:42.603478 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:17:42.603605 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:17:42.603676 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.603768 (+ 92us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.603823 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 21:17:42.605527 31170 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:17:42.607012 31170 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:17:42.613350 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.609217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38566 (local address 127.20.115.194:33807)
0924 21:17:42.609417 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.609422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.609431 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:42.609469 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.609471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.609492 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.609575 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:42.609747 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.610721 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.612166 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.612912 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.612980 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.613030 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.613212 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.613258 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":104,"threads_started":1}
I20260924 21:17:42.613369 31638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.609043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:44059 (local address 127.0.0.1:45276)
0924 21:17:42.609892 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:42.609900 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:42.609949 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:42.610116 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:42.610117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:42.610119 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:42.610157 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.610158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.611116 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.611117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.612287 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.612290 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.613010 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.613011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.613097 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.613112 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 21:17:42.613186 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 21:17:42.613259 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":27,"thread_start_us":52,"threads_started":1}
I20260924 21:17:42.613662 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.609252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45276 (local address 127.20.115.195:44059)
0924 21:17:42.609763 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.609767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.609942 (+ 175us) server_negotiation.cc:407] Connection header received
0924 21:17:42.610025 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.610027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.610042 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.610104 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:42.610212 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.611019 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.612373 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.612927 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.613013 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.613077 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.613536 (+ 459us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.613587 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":72,"threads_started":1}
I20260924 21:17:42.613369 31639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.609036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:33807 (local address 127.0.0.1:38566)
0924 21:17:42.609206 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:42.609245 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:42.609310 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:42.609577 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:42.609580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:42.609581 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:42.609644 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.609646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.610816 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.610817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.612073 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.612076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.613062 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.613063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.613121 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.613124 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:42.613185 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:17:42.613247 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260924 21:17:42.614396 31563 tablet_service.cc:1467] Processing CreateTablet for tablet de37aa77ce094a5b85f3cec9f5abefdf (DEFAULT_TABLE table=test-table [id=f8a7d3edf34e483ba3c7999a97a68f45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:42.614418 31429 tablet_service.cc:1467] Processing CreateTablet for tablet de37aa77ce094a5b85f3cec9f5abefdf (DEFAULT_TABLE table=test-table [id=f8a7d3edf34e483ba3c7999a97a68f45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:42.614593 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.609039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45455 (local address 127.0.0.1:46112)
0924 21:17:42.609609 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:42.609618 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:42.609681 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:42.610934 (+ 1253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:42.610935 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:42.610937 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:42.610980 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.610982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.612252 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.612253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.613321 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.613323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.614354 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.614355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.614417 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.614420 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:42.614476 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 21:17:42.614520 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":2,"thread_start_us":58,"threads_started":1}
I20260924 21:17:42.614710 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.609426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46112 (local address 127.20.115.193:45455)
0924 21:17:42.609579 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.609584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.609688 (+ 104us) server_negotiation.cc:407] Connection header received
0924 21:17:42.610834 (+ 1146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.610836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.610850 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.610935 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:42.611046 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.612161 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.613410 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.614250 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.614354 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.614402 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.614489 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.614525 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260924 21:17:42.614770 31429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de37aa77ce094a5b85f3cec9f5abefdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:42.614770 31563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de37aa77ce094a5b85f3cec9f5abefdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:42.615330 31295 tablet_service.cc:1467] Processing CreateTablet for tablet de37aa77ce094a5b85f3cec9f5abefdf (DEFAULT_TABLE table=test-table [id=f8a7d3edf34e483ba3c7999a97a68f45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:17:42.615603 31295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de37aa77ce094a5b85f3cec9f5abefdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:17:42.618108 31643 tablet_bootstrap.cc:492] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5: Bootstrap starting.
I20260924 21:17:42.618443 31644 tablet_bootstrap.cc:492] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: Bootstrap starting.
I20260924 21:17:42.618777 31643 tablet_bootstrap.cc:654] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:42.618753 31645 tablet_bootstrap.cc:492] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: Bootstrap starting.
I20260924 21:17:42.619096 31644 tablet_bootstrap.cc:654] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:42.619313 31643 log.cc:824] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:42.619390 31645 tablet_bootstrap.cc:654] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: Neither blocks nor log segments found. Creating new log.
I20260924 21:17:42.619642 31644 log.cc:824] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:42.619975 31645 log.cc:824] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: Log is configured to *not* fsync() on all Append() calls
I20260924 21:17:42.620474 31644 tablet_bootstrap.cc:492] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: No bootstrap required, opened a new log
I20260924 21:17:42.620548 31644 ts_tablet_manager.cc:1397] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:17:42.620694 31645 tablet_bootstrap.cc:492] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: No bootstrap required, opened a new log
I20260924 21:17:42.620754 31645 ts_tablet_manager.cc:1397] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:17:42.622630 31645 raft_consensus.cc:348] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } }
I20260924 21:17:42.622591 31644 raft_consensus.cc:348] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } }
I20260924 21:17:42.622763 31644 raft_consensus.cc:374] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:42.622763 31645 raft_consensus.cc:374] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:42.622799 31645 raft_consensus.cc:729] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a58871ffa6a4054aee58e16b67589f9, State: Initialized, Role: FOLLOWER
I20260924 21:17:42.622799 31644 raft_consensus.cc:729] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfdfa9eee7184ba4b30a5a6b4ea86fed, State: Initialized, Role: FOLLOWER
I20260924 21:17:42.622920 31645 consensus_queue.cc:260] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } }
I20260924 21:17:42.622947 31644 consensus_queue.cc:260] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } }
I20260924 21:17:42.623203 31644 ts_tablet_manager.cc:1428] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:17:42.623203 31645 ts_tablet_manager.cc:1428] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:17:42.623241 31495 heartbeater.cc:503] Master 127.20.115.254:44567 was elected leader, sending a full tablet report...
I20260924 21:17:42.623315 31629 heartbeater.cc:503] Master 127.20.115.254:44567 was elected leader, sending a full tablet report...
I20260924 21:17:42.623629 31643 tablet_bootstrap.cc:492] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5: No bootstrap required, opened a new log
I20260924 21:17:42.623696 31643 ts_tablet_manager.cc:1397] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260924 21:17:42.625360 31643 raft_consensus.cc:348] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } }
I20260924 21:17:42.625481 31643 raft_consensus.cc:374] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:17:42.625502 31643 raft_consensus.cc:729] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36260c417e3749aea1d75096eaad61c5, State: Initialized, Role: FOLLOWER
I20260924 21:17:42.625609 31643 consensus_queue.cc:260] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } }
I20260924 21:17:42.625833 31643 ts_tablet_manager.cc:1428] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:17:42.625938 31361 heartbeater.cc:503] Master 127.20.115.254:44567 was elected leader, sending a full tablet report...
W20260924 21:17:42.678841 31496 tablet.cc:2369] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:17:42.679123 31651 raft_consensus.cc:482] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:17:42.679286 31651 raft_consensus.cc:504] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } }
I20260924 21:17:42.679730 31651 leader_election.cc:288] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a58871ffa6a4054aee58e16b67589f9 (127.20.115.194:33807), cfdfa9eee7184ba4b30a5a6b4ea86fed (127.20.115.195:44059)
I20260924 21:17:42.683526 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.679926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:44059 (local address 127.20.115.193:56607)
0924 21:17:42.679991 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:42.680002 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:42.680043 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:42.680323 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:42.680325 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:42.680327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:42.680436 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.680440 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.681430 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.681431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.682637 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.682640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.683239 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.683240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.683322 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.683325 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:17:42.683410 (+ 85us) client_negotiation.cc:240] Negotiation successful
0924 21:17:42.683449 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":46}
I20260924 21:17:42.683526 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.680118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:56607 (local address 127.20.115.195:44059)
0924 21:17:42.680169 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.680171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.680180 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:17:42.680210 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.680211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.680222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.680280 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:42.680516 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.681332 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.682738 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.683131 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.683244 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.683255 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.683419 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.683459 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:17:42.684684 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.679927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:33807 (local address 127.20.115.193:39895)
0924 21:17:42.680209 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:42.680219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:17:42.680277 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:17:42.680399 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:17:42.680400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:17:42.680402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:17:42.680472 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.680474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.681408 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.681409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.683915 (+ 2506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:17:42.683918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.684427 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.684428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:17:42.684499 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.684504 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:17:42.684572 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 21:17:42.684611 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":22,"thread_start_us":100,"threads_started":1}
I20260924 21:17:42.684713 31640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.680093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:39895 (local address 127.20.115.194:33807)
0924 21:17:42.680140 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.680143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.680289 (+ 146us) server_negotiation.cc:407] Connection header received
0924 21:17:42.680325 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.680326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.680336 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.680397 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:17:42.680580 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.681296 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.684000 (+ 2704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.684347 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.684406 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.684411 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.684610 (+ 199us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.684651 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:17:42.685025 31449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de37aa77ce094a5b85f3cec9f5abefdf" candidate_uuid: "36260c417e3749aea1d75096eaad61c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a58871ffa6a4054aee58e16b67589f9" is_pre_election: true
I20260924 21:17:42.685181 31449 raft_consensus.cc:2454] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36260c417e3749aea1d75096eaad61c5 in term 0.
I20260924 21:17:42.685434 31251 leader_election.cc:302] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36260c417e3749aea1d75096eaad61c5, 3a58871ffa6a4054aee58e16b67589f9; no voters:
I20260924 21:17:42.685596 31651 raft_consensus.cc:2781] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:17:42.685670 31651 raft_consensus.cc:482] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:17:42.685724 31651 raft_consensus.cc:3037] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:42.686717 31651 raft_consensus.cc:504] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } }
I20260924 21:17:42.686858 31651 leader_election.cc:288] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [CANDIDATE]: Term 1 election: Requested vote from peers 3a58871ffa6a4054aee58e16b67589f9 (127.20.115.194:33807), cfdfa9eee7184ba4b30a5a6b4ea86fed (127.20.115.195:44059)
I20260924 21:17:42.687060 31449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de37aa77ce094a5b85f3cec9f5abefdf" candidate_uuid: "36260c417e3749aea1d75096eaad61c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a58871ffa6a4054aee58e16b67589f9"
I20260924 21:17:42.687158 31449 raft_consensus.cc:3037] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:42.688212 31449 raft_consensus.cc:2454] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36260c417e3749aea1d75096eaad61c5 in term 1.
I20260924 21:17:42.688369 31251 leader_election.cc:302] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36260c417e3749aea1d75096eaad61c5, 3a58871ffa6a4054aee58e16b67589f9; no voters:
I20260924 21:17:42.688490 31651 raft_consensus.cc:2781] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:17:42.688668 31651 raft_consensus.cc:686] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [term 1 LEADER]: Becoming Leader. State: Replica: 36260c417e3749aea1d75096eaad61c5, State: Running, Role: LEADER
I20260924 21:17:42.688759 31651 consensus_queue.cc:237] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } }
I20260924 21:17:42.689634 31169 catalog_manager.cc:5495] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 36260c417e3749aea1d75096eaad61c5 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "36260c417e3749aea1d75096eaad61c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36260c417e3749aea1d75096eaad61c5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 } health_report { overall_health: UNKNOWN } } }
I20260924 21:17:42.693207 31582 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de37aa77ce094a5b85f3cec9f5abefdf" candidate_uuid: "36260c417e3749aea1d75096eaad61c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed"
I20260924 21:17:42.693338 31582 raft_consensus.cc:3037] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:17:42.693681 31583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de37aa77ce094a5b85f3cec9f5abefdf" candidate_uuid: "36260c417e3749aea1d75096eaad61c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" is_pre_election: true
I20260924 21:17:42.694346 31582 raft_consensus.cc:2454] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36260c417e3749aea1d75096eaad61c5 in term 1.
I20260924 21:17:42.704135 31642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.702008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46120 (local address 127.20.115.193:45455)
0924 21:17:42.702072 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 21:17:42.702074 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:17:42.702086 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:17:42.702157 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:17:42.702158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:17:42.702173 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:17:42.702231 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:17:42.702335 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.702783 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.703475 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:17:42.703656 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:17:42.703705 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:17:42.703836 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:17:42.703923 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:17:42.703981 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:17:42.704019 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 21:17:42.704034 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:17:42.705808 31582 raft_consensus.cc:1261] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed [term 1 FOLLOWER]: Refusing update from remote peer 36260c417e3749aea1d75096eaad61c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:42.705864 31449 raft_consensus.cc:1261] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [term 1 FOLLOWER]: Refusing update from remote peer 36260c417e3749aea1d75096eaad61c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:17:42.706159 31653 consensus_queue.cc:1035] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfdfa9eee7184ba4b30a5a6b4ea86fed" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:42.706293 31651 consensus_queue.cc:1035] T de37aa77ce094a5b85f3cec9f5abefdf P 36260c417e3749aea1d75096eaad61c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a58871ffa6a4054aee58e16b67589f9" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:17:42.708442 31656 mvcc.cc:205] Tried to move back new op lower bound from 7333005978440474624 to 7333005978373763072. Current Snapshot: MvccSnapshot[applied={T|T < 7333005978440474624}]
I20260924 21:17:42.708563 31658 mvcc.cc:205] Tried to move back new op lower bound from 7333005978440474624 to 7333005978373763072. Current Snapshot: MvccSnapshot[applied={T|T < 7333005978440474624}]
I20260924 21:17:42.708683 31657 mvcc.cc:205] Tried to move back new op lower bound from 7333005978440474624 to 7333005978373763072. Current Snapshot: MvccSnapshot[applied={T|T < 7333005978440474624}]
I20260924 21:17:42.710825 31170 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:42.710912 31170 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:17:42.711639 31170 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 98c052d533f44dfaba3976eb9f9153b0: Sending DeleteTablet for 3 replicas of tablet de37aa77ce094a5b85f3cec9f5abefdf
I20260924 21:17:42.711951 31429 tablet_service.cc:1514] Processing DeleteTablet for tablet de37aa77ce094a5b85f3cec9f5abefdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:42 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:38566
I20260924 21:17:42.711993 31563 tablet_service.cc:1514] Processing DeleteTablet for tablet de37aa77ce094a5b85f3cec9f5abefdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:42 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:45276
I20260924 21:17:42.712186 31666 tablet_replica.cc:330] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: stopping tablet replica
I20260924 21:17:42.712219 31295 tablet_service.cc:1514] Processing DeleteTablet for tablet de37aa77ce094a5b85f3cec9f5abefdf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:17:42 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:46112
I20260924 21:17:42.712286 31666 raft_consensus.cc:2229] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:42.712394 31666 raft_consensus.cc:2258] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:42.712543 31667 tablet_replica.cc:330] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: stopping tablet replica
I20260924 21:17:42.712817 31667 raft_consensus.cc:2229] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:17:42.712824 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 31232
I20260924 21:17:42.713019 31666 ts_tablet_manager.cc:1905] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:42.713474 31667 raft_consensus.cc:2258] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:17:42.714241 31667 ts_tablet_manager.cc:1905] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:17:42.715273 31666 ts_tablet_manager.cc:1918] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:42.715348 31666 log.cc:1196] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/de37aa77ce094a5b85f3cec9f5abefdf
I20260924 21:17:42.715638 31666 ts_tablet_manager.cc:1939] T de37aa77ce094a5b85f3cec9f5abefdf P 3a58871ffa6a4054aee58e16b67589f9: Deleting consensus metadata
I20260924 21:17:42.716042 31157 catalog_manager.cc:4841] TS 3a58871ffa6a4054aee58e16b67589f9 (127.20.115.194:33807): tablet de37aa77ce094a5b85f3cec9f5abefdf (table test-table [id=f8a7d3edf34e483ba3c7999a97a68f45]) successfully deleted
I20260924 21:17:42.716236 31667 ts_tablet_manager.cc:1918] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:17:42.716307 31667 log.cc:1196] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/de37aa77ce094a5b85f3cec9f5abefdf
I20260924 21:17:42.716571 31667 ts_tablet_manager.cc:1939] T de37aa77ce094a5b85f3cec9f5abefdf P cfdfa9eee7184ba4b30a5a6b4ea86fed: Deleting consensus metadata
I20260924 21:17:42.716955 31156 catalog_manager.cc:4841] TS cfdfa9eee7184ba4b30a5a6b4ea86fed (127.20.115.195:44059): tablet de37aa77ce094a5b85f3cec9f5abefdf (table test-table [id=f8a7d3edf34e483ba3c7999a97a68f45]) successfully deleted
W20260924 21:17:42.718940 31155 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:45455 (error 108)
I20260924 21:17:42.719156 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 31366
I20260924 21:17:42.719731 31637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:17:42.719256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45455 (local address 127.0.0.1:46122)
0924 21:17:42.719654 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0924 21:17:42.719692 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378}
W20260924 21:17:42.719964 31155 catalog_manager.cc:4584] TS 36260c417e3749aea1d75096eaad61c5 (127.20.115.193:45455): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de37aa77ce094a5b85f3cec9f5abefdf: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45455: connect: Connection refused (error 111)
I20260924 21:17:42.723722 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 31500
I20260924 21:17:42.729220 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 31138
2026-09-24T21:17:42Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1909 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31672](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31672](info): set up 2 sockets
Sep 24 21:17:42 dist-test-slave-cxpd krb5kdc[31672](info): commencing operation
krb5kdc: starting...
W20260924 21:17:43.761657 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:17:43 dist-test-slave-cxpd krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:17:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:17:43Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/rangeradmin_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/rangeradmin_127.20.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/rangerlookup_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/rangerlookup_127.20.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/HTTP_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/HTTP_127.20.115.212@KRBTEST.COM.keytab.
I20260924 21:17:43.931706 20943 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNZGLg7/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/postgres -l logfile start
2026-09-24 21:17:46.572 UTC [31702] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 21:17:46.573 UTC [31702] LOG: listening on IPv4 address "127.20.115.212", port 55251
2026-09-24 21:17:46.577 UTC [31702] LOG: listening on Unix socket "/tmp/.s.PGSQL.55251"
2026-09-24 21:17:46.597 UTC [31705] LOG: database system was shut down at 2026-09-24 21:17:44 UTC
2026-09-24 21:17:46.602 UTC [31702] LOG: database system is ready to accept connections
I20260924 21:17:47.577633 20943 mini_postgres.cc:96] Postgres bound to 55251
2026-09-24 21:17:47.583 UTC [31713] FATAL: database "slave" does not exist
127.20.115.212:55251 - accepting connections
I20260924 21:17:47.584106 20943 mini_ranger.cc:161] Starting Ranger...
I20260924 21:17:47.598605 20943 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 21:17:48.129320 20943 mini_ranger.cc:87] Created ranger Postgres database
I20260924 21:17:48.129419 20943 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-admin
2026-09-24 21:17:48,425 [I] DB FLAVOR :POSTGRES
2026-09-24 21:17:48,425 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 21:17:48,425 [I] Checking connection..
2026-09-24 21:17:48,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 21:17:48,656 [I] Checking connection passed.
2026-09-24 21:17:48,657 [I] --------- Verifying version history table ---------
2026-09-24 21:17:48,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:17:48,918 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 21:17:48,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:17:49,172 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 21:17:49,173 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 21:17:49,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 21:17:49.421 UTC [31777] WARNING: there is no transaction in progress
2026-09-24 21:17:49,428 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 21:17:49,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:17:49,677 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 21:17:49,678 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 21:17:49,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 21:17:49,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 21:17:50,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cxpd.c.gcp-upstream.internal','N') ;"
2026-09-24 21:17:50,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 21:17:50,630 [I] Table x_portal_user does not exist in database ranger
2026-09-24 21:17:50,630 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 21:17:50,873 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 21:17:50,873 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 21:17:51,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 21:17:51,351 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 21:17:51,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 21:17:52.329 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.347 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.363 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.379 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.395 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.416 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.480 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.487 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.502 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.512 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.523 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.538 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.547 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.554 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.562 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:52.569 UTC [31885] WARNING: there is no transaction in progress
2026-09-24 21:17:53,248 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 21:17:53,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 21:17:53,467 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 21:17:53,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 21:17:53,701 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 21:17:53,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 21:17:53,948 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 21:17:53,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 21:17:54,178 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 21:17:54,403 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cxpd.c.gcp-upstream.internal';"
2026-09-24 21:17:54,620 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 21:17:54,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 21:17:54,862 [I] DB_PATCHES have already been applied
I20260924 21:17:54.868613 20943 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNZGLg7/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 21:17:54.868686 20943 mini_ranger.cc:193] Using host: 127.20.115.212
I20260924 21:17:54.871917 20943 mini_ranger.cc:227] Ranger admin URL: http://127.20.115.212:44699
Sep 24, 2026 9:17:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 9:17:55 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 9:17:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 9:17:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 9:17:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 9:17:55 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 9:17:55 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 21:17:55.971040 20943 mini_ranger.cc:160] Time spent starting Ranger: real 8.387s user 0.000s sys 0.009s
Sep 24, 2026 9:18:15 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 9:18:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 24, 2026 9:18:15 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 9:18:15 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 9:18:16 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 9:18:16 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 21:18:17.375073 20943 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/rangerkms_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/rangerkms_127.20.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.115.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.115.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.115.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/HTTP_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.115.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/HTTP_127.20.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 21:18:17 dist-test-slave-cxpd krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284697, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 21:18:17.470719 20943 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 21:18:17.480468 20943 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 21:18:18.451051 20943 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 21:18:18.451120 20943 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-kms
I20260924 21:18:18.451133 20943 mini_ranger_kms.cc:227] Using postgres at 127.20.115.212:55251
2026-09-24 21:18:18,614 [I] DB FLAVOR :POSTGRES
2026-09-24 21:18:18,615 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 21:18:18,615 [I] Checking connection
2026-09-24 21:18:18,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 21:18:18,868 [I] connection success
2026-09-24 21:18:18,869 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 21:18:18,869 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 21:18:18,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 21:18:19,125 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 21:18:19,125 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 21:18:19,125 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 21:18:19,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:55251/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 21:18:19,402 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 21:18:19.892190 20943 mini_ranger_kms.cc:310] Created kms service
I20260924 21:18:20.082787 20943 mini_ranger_kms.cc:326] Created kudu user
I20260924 21:18:20.131253 20943 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 21:18:20 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284697, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:18:20.560153 20943 mini_ranger_kms.cc:384] Added ranger policy
I20260924 21:18:20.560267 20943 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 21:18:20.560292 20943 mini_ranger_kms.cc:240] Using host: 127.20.115.212
I20260924 21:18:20.563656 20943 mini_ranger_kms.cc:276] Ranger KMS PID: 32065
I20260924 21:18:20.563781 20943 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.115.212:59131
Sep 24, 2026 9:18:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 9:18:21 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 9:18:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 9:18:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 9:18:22 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 9:18:22 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 21:18:22.824878 20943 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.354s user 0.001s sys 0.146s
I20260924 21:18:22.824990 20943 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 21:18:22.825074 20943 mini_ranger_kms.cc:401] 127.20.115.212:59131/kms/v1/keys
Sep 24 21:18:22 dist-test-slave-cxpd krb5kdc[31672](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284702, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.115.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 21:18:23 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284702, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.115.212@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
Sep 24 21:18:23 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284702, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.115.212@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
Sep 24 21:18:25 dist-test-slave-cxpd krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:18:25.905992 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43809
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:43809
--ranger_config_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:59131/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:18:25.985605 32116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:18:25.985793 32116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:18:25.985814 32116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:18:25.986948 32116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:18:25.986986 32116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:18:25.987000 32116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:18:25.987013 32116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:18:25.987072 32116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:18:25.988624 32116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.115.212:59131/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:43809
--ranger_config_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:18:25.988898 32116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:18:25.989135 32116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:18:25.994163 32116 server_base.cc:1034] running on GCE node
W20260924 21:18:25.994208 32122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:18:25.994122 32121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:18:25.994133 32124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:18:25.994577 32116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:18:25.994884 32116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:18:25.996053 32116 hybrid_clock.cc:648] HybridClock initialized: now 1790284705996047 us; error 48 us; skew 500 ppm
Sep 24 21:18:26 dist-test-slave-cxpd krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284706, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:18:26.003696 32116 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:18:26.004274 32116 webserver.cc:460] Webserver started at http://127.20.115.254:40685/ using document root <none> and password file <none>
I20260924 21:18:26.004531 32116 fs_manager.cc:359] Metadata directory not provided
I20260924 21:18:26.004588 32116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:18:26.004716 32116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:18:26 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284706, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:18:26.061340 32116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "00073874a08b42c1bdc04b3c1969ae3f"
format_stamp: "Formatted at 2026-09-24 21:18:26 on dist-test-slave-cxpd"
server_key: "f1db744403314c3081bb27526d61576f"
server_key_iv: "ce273d918dfbcd9ff49cfb73a12c0762"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:26.061817 32116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "00073874a08b42c1bdc04b3c1969ae3f"
format_stamp: "Formatted at 2026-09-24 21:18:26 on dist-test-slave-cxpd"
server_key: "f1db744403314c3081bb27526d61576f"
server_key_iv: "ce273d918dfbcd9ff49cfb73a12c0762"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:26.063584 32116 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260924 21:18:26.078148 32131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:26.078398 32116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:18:26.078501 32116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "00073874a08b42c1bdc04b3c1969ae3f"
format_stamp: "Formatted at 2026-09-24 21:18:26 on dist-test-slave-cxpd"
server_key: "f1db744403314c3081bb27526d61576f"
server_key_iv: "ce273d918dfbcd9ff49cfb73a12c0762"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:26.078608 32116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:18:26.095077 32116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:18:26.097189 32116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:18:26.097311 32116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:18:26.101429 32116 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:43809
I20260924 21:18:26.101505 32183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:43809 every 8 connection(s)
I20260924 21:18:26.101840 32116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:18:26.102484 32184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:18:26.104696 32184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f: Bootstrap starting.
I20260924 21:18:26.105381 32184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f: Neither blocks nor log segments found. Creating new log.
I20260924 21:18:26.105683 32184 log.cc:824] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f: Log is configured to *not* fsync() on all Append() calls
I20260924 21:18:26.106338 32184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f: No bootstrap required, opened a new log
I20260924 21:18:26.108343 32184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00073874a08b42c1bdc04b3c1969ae3f" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43809 } }
I20260924 21:18:26.108495 32184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:18:26.108518 32184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00073874a08b42c1bdc04b3c1969ae3f, State: Initialized, Role: FOLLOWER
I20260924 21:18:26.108667 32184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00073874a08b42c1bdc04b3c1969ae3f" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43809 } }
I20260924 21:18:26.108747 32184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:18:26.108771 32184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:18:26.108853 32184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:18:26.109372 32184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00073874a08b42c1bdc04b3c1969ae3f" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43809 } }
I20260924 21:18:26.109467 32184 leader_election.cc:302] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00073874a08b42c1bdc04b3c1969ae3f; no voters:
I20260924 21:18:26.109652 32184 leader_election.cc:288] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:18:26.109694 32187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:18:26.109891 32187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [term 1 LEADER]: Becoming Leader. State: Replica: 00073874a08b42c1bdc04b3c1969ae3f, State: Running, Role: LEADER
I20260924 21:18:26.109944 32184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:18:26.110023 32187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00073874a08b42c1bdc04b3c1969ae3f" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43809 } }
I20260924 21:18:26.110387 32189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00073874a08b42c1bdc04b3c1969ae3f. Latest consensus state: current_term: 1 leader_uuid: "00073874a08b42c1bdc04b3c1969ae3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00073874a08b42c1bdc04b3c1969ae3f" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43809 } } }
I20260924 21:18:26.110365 32188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00073874a08b42c1bdc04b3c1969ae3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00073874a08b42c1bdc04b3c1969ae3f" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43809 } } }
I20260924 21:18:26.110456 32189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [sys.catalog]: This master's current role is: LEADER
I20260924 21:18:26.110463 32188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f [sys.catalog]: This master's current role is: LEADER
I20260924 21:18:26.110679 32194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:18:26.111243 32194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:18:26.111493 32184 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 21:18:26.112038 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 32116
I20260924 21:18:26.112171 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:18:26.112371 20943 external_mini_cluster.cc:1428] Setting key dbf15e6e291b661aab910d78474b7d45
I20260924 21:18:26.116482 32194 catalog_manager.cc:1347] Generated new cluster ID: 229f0dfe8fa849939d61b42b6b74f91f
I20260924 21:18:26.116549 32194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:18:26 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:18:26.128557 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:26.113854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.20.115.254:43809)
0924 21:18:26.114336 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:26.114351 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:26.114415 (+ 64us) server_negotiation.cc:407] Connection header received
0924 21:18:26.114756 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:26.114769 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:26.114899 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:26.115074 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:18:26.116241 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:26.117720 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:26.118938 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:26.119572 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:26.119682 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:26.124886 (+ 5204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:18:26.124891 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:18:26.124891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:18:26.124903 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:18:26.126987 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:26.127369 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:26.127372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:26.127372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:26.127482 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:26.127663 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:26.127664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:26.127664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:26.127970 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:18:26.128119 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:26.128210 (+ 91us) server_negotiation.cc:299] Negotiation successful
0924 21:18:26.128328 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":204,"threads_started":1}
I20260924 21:18:26.141605 32194 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:18:26.142130 32194 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:18:26.149849 32194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f: Generated new TSK 0
I20260924 21:18:26.150045 32194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:18:26.852335 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:59131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43809
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:18:26.965471 32229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:18:26.965700 32229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:18:26.965734 32229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:18:26.966943 32229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:18:26.967001 32229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:18:26.967046 32229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:18:26.968488 32229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.115.212:59131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:18:26.968744 32229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:18:26.969070 32229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:18:26.974210 32237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:18:26.974217 32235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:18:26.974567 32229 server_base.cc:1034] running on GCE node
W20260924 21:18:26.974699 32234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:18:26.974929 32229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:18:26.975189 32229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:18:26.976362 32229 hybrid_clock.cc:648] HybridClock initialized: now 1790284706976349 us; error 42 us; skew 500 ppm
Sep 24 21:18:26 dist-test-slave-cxpd krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284706, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:18:26.979151 32229 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:18:26.979712 32229 webserver.cc:460] Webserver started at http://127.20.115.193:33631/ using document root <none> and password file <none>
I20260924 21:18:26.980020 32229 fs_manager.cc:359] Metadata directory not provided
I20260924 21:18:26.980118 32229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:18:26.980284 32229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:18:26 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284706, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:18:26.993352 32229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "c369b75efded4c30bb97fc9e7d02f689"
format_stamp: "Formatted at 2026-09-24 21:18:26 on dist-test-slave-cxpd"
server_key: "a92cd2028cfb74d118cbb17145c6de98"
server_key_iv: "da5c51b321fc6ec566e520381a28ca4e"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:26.993927 32229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "c369b75efded4c30bb97fc9e7d02f689"
format_stamp: "Formatted at 2026-09-24 21:18:26 on dist-test-slave-cxpd"
server_key: "a92cd2028cfb74d118cbb17145c6de98"
server_key_iv: "da5c51b321fc6ec566e520381a28ca4e"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:26.997195 32229 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260924 21:18:27.012641 32246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:27.012948 32229 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:18:27.013051 32229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "c369b75efded4c30bb97fc9e7d02f689"
format_stamp: "Formatted at 2026-09-24 21:18:26 on dist-test-slave-cxpd"
server_key: "a92cd2028cfb74d118cbb17145c6de98"
server_key_iv: "da5c51b321fc6ec566e520381a28ca4e"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:27.013170 32229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:18:27.044025 32229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:18:27.046615 32229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:18:27.046829 32229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:18:27.047135 32229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:18:27.047521 32229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:18:27.047601 32229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:27.047698 32229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:18:27.047773 32229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:27.054986 32229 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:36329
I20260924 21:18:27.055521 32229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:18:27.059227 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 32229
I20260924 21:18:27.059340 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:18:27.059568 20943 external_mini_cluster.cc:1428] Setting key 8306f828a6d15efb32e19b5b6fecf4b2
I20260924 21:18:27.058588 32359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:36329 every 8 connection(s)
Sep 24 21:18:27 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284706, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
I20260924 21:18:27.077207 32363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.058215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43809 (local address 127.20.115.193:48245)
0924 21:18:27.058739 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0924 21:18:27.058781 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:18:27.059476 (+ 695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:18:27.067294 (+ 7818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:18:27.067300 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:18:27.067550 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:18:27.067833 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.067839 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.068952 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.068953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.070230 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.070233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.070789 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.070790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.070962 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.071633 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:18:27.071645 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:18:27.073366 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:18:27.075513 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:18:27.075520 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:18:27.075523 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:18:27.075820 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:18:27.076081 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:18:27.076082 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:18:27.076082 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:18:27.076210 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:18:27.076520 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:18:27.076526 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:18:27.076682 (+ 156us) client_negotiation.cc:769] Sending connection context
0924 21:18:27.076772 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 21:18:27.076952 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":76,"threads_started":1}
I20260924 21:18:27.077821 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.058209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:48245 (local address 127.20.115.254:43809)
0924 21:18:27.067019 (+ 8810us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.067021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.067032 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:18:27.067061 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.067061 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.067075 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.067946 (+ 871us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:18:27.067991 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.068819 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.070331 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.070686 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.073461 (+ 2775us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.073516 (+ 55us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:18:27.073518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:18:27.073518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:18:27.073523 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:18:27.075388 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:27.075921 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:27.075922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:27.075923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:27.075990 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:27.076301 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:27.076302 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:27.076303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:27.076439 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:18:27.077594 (+ 1155us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.077637 (+ 43us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.077725 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8777,"thread_start_us":85,"threads_started":1}
I20260924 21:18:27.079100 32360 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43809
I20260924 21:18:27.079193 32360 heartbeater.cc:464] Registering TS with master...
I20260924 21:18:27.079375 32360 heartbeater.cc:511] Master 127.20.115.254:43809 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
I20260924 21:18:27.079998 32148 ts_manager.cc:194] Registered new tserver with Master: c369b75efded4c30bb97fc9e7d02f689 (127.20.115.193:36329)
I20260924 21:18:27.080904 32148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:48245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:18:27.097754 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:59131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43809
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 21:18:27 dist-test-slave-cxpd krb5kdc[31672](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284707, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260924 21:18:27.216233 32368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:18:27.216495 32368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:18:27.216526 32368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:18:27.218169 32368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:18:27.218219 32368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:18:27.218256 32368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:18:27.220268 32368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.115.212:59131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:18:27.220490 32368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:18:27.220721 32368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:18:27.226094 32373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:18:27.226101 32374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:18:27.226270 32368 server_base.cc:1034] running on GCE node
W20260924 21:18:27.226536 32376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:18:27.226780 32368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:18:27.227092 32368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:18:27.228272 32368 hybrid_clock.cc:648] HybridClock initialized: now 1790284707228241 us; error 43 us; skew 500 ppm
Sep 24 21:18:27 dist-test-slave-cxpd krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284707, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:18:27.231482 32368 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:18:27.231956 32368 webserver.cc:460] Webserver started at http://127.20.115.194:43413/ using document root <none> and password file <none>
I20260924 21:18:27.232182 32368 fs_manager.cc:359] Metadata directory not provided
I20260924 21:18:27.232232 32368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:18:27.232348 32368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:18:27 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284707, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:18:27.247803 32368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "8eeadd2b5ff648e99e576a9c2ec364a4"
format_stamp: "Formatted at 2026-09-24 21:18:27 on dist-test-slave-cxpd"
server_key: "77233ce16ca8d7df7b81a13fa642bd49"
server_key_iv: "128733e4a6dc4ee0f24f72f34f65a307"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:27.248241 32368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "8eeadd2b5ff648e99e576a9c2ec364a4"
format_stamp: "Formatted at 2026-09-24 21:18:27 on dist-test-slave-cxpd"
server_key: "77233ce16ca8d7df7b81a13fa642bd49"
server_key_iv: "128733e4a6dc4ee0f24f72f34f65a307"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:27.250376 32368 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260924 21:18:27.272941 32384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:27.273181 32368 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.001s sys 0.000s
I20260924 21:18:27.273279 32368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "8eeadd2b5ff648e99e576a9c2ec364a4"
format_stamp: "Formatted at 2026-09-24 21:18:27 on dist-test-slave-cxpd"
server_key: "77233ce16ca8d7df7b81a13fa642bd49"
server_key_iv: "128733e4a6dc4ee0f24f72f34f65a307"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:27.273381 32368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:18:27.310042 32368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:18:27.312635 32368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:18:27.312875 32368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:18:27.313199 32368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:18:27.313606 32368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:18:27.313696 32368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:27.313771 32368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:18:27.313836 32368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:27.320345 32368 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:33521
I20260924 21:18:27.320844 32368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:18:27.323663 32497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:33521 every 8 connection(s)
I20260924 21:18:27.328008 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 32368
I20260924 21:18:27.328140 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:18:27.328330 20943 external_mini_cluster.cc:1428] Setting key 5d0916cb4682fdf551ab8b158c689763
Sep 24 21:18:27 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284707, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:18:27.335227 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.324749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:39567 (local address 127.20.115.254:43809)
0924 21:18:27.324816 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.324818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.325098 (+ 280us) server_negotiation.cc:407] Connection header received
0924 21:18:27.325725 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.325726 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.325742 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.325812 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:18:27.326552 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.327358 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.328915 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.329228 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.329278 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.331862 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:18:27.331864 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:18:27.331865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:18:27.331872 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:18:27.333644 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:27.334147 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:27.334148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:27.334148 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:27.334205 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:27.334473 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:27.334474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:27.334475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:27.334605 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:18:27.334646 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.335036 (+ 390us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.335133 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:18:27.336040 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.324716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43809 (local address 127.20.115.194:39567)
0924 21:18:27.324924 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0924 21:18:27.324979 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0924 21:18:27.325592 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:18:27.325900 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:18:27.325904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:18:27.326160 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:18:27.326435 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.326443 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.327473 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.327475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.328810 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.328813 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.329375 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.329375 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.329518 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.330126 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:18:27.330136 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:18:27.331748 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:18:27.333744 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:18:27.333750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:18:27.333752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:18:27.334054 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:18:27.334277 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:18:27.334278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:18:27.334278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:18:27.334399 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:18:27.334690 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:18:27.334695 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:18:27.334846 (+ 151us) client_negotiation.cc:769] Sending connection context
0924 21:18:27.335637 (+ 791us) client_negotiation.cc:240] Negotiation successful
0924 21:18:27.335799 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
I20260924 21:18:27.340880 32498 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43809
I20260924 21:18:27.341045 32498 heartbeater.cc:464] Registering TS with master...
I20260924 21:18:27.341292 32498 heartbeater.cc:511] Master 127.20.115.254:43809 requested a full tablet report, sending...
I20260924 21:18:27.341706 32148 ts_manager.cc:194] Registered new tserver with Master: 8eeadd2b5ff648e99e576a9c2ec364a4 (127.20.115.194:33521)
I20260924 21:18:27.342209 32148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:39567
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:18:27.377907 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:59131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43809
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:18:27.474720 32503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:18:27.475023 32503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:18:27.475060 32503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:18:27.476745 32503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:18:27.476827 32503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:18:27.476866 32503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:18:27.479094 32503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.115.212:59131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:18:27.479342 32503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:18:27.479607 32503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:18:27.489650 32509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:18:27.489796 32503 server_base.cc:1034] running on GCE node
W20260924 21:18:27.489894 32511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:18:27.490082 32508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:18:27.490331 32503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:18:27.490619 32503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:18:27.491797 32503 hybrid_clock.cc:648] HybridClock initialized: now 1790284707491769 us; error 43 us; skew 500 ppm
Sep 24 21:18:27 dist-test-slave-cxpd krb5kdc[31672](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284707, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:18:27.497494 32503 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:18:27.498096 32503 webserver.cc:460] Webserver started at http://127.20.115.195:46201/ using document root <none> and password file <none>
I20260924 21:18:27.498379 32503 fs_manager.cc:359] Metadata directory not provided
I20260924 21:18:27.498446 32503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:18:27.498582 32503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:18:27 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284707, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:18:27.523255 32503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "e92aa07aaee74cd3b83a6655cb7859d2"
format_stamp: "Formatted at 2026-09-24 21:18:27 on dist-test-slave-cxpd"
server_key: "e1bfd7ac55e9008939ecdb59804875f8"
server_key_iv: "0719e8fa74d0f9c1857a18aa9abbf8e0"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:27.523736 32503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "e92aa07aaee74cd3b83a6655cb7859d2"
format_stamp: "Formatted at 2026-09-24 21:18:27 on dist-test-slave-cxpd"
server_key: "e1bfd7ac55e9008939ecdb59804875f8"
server_key_iv: "0719e8fa74d0f9c1857a18aa9abbf8e0"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:27.527244 32503 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260924 21:18:27.544953 32518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:27.545223 32503 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260924 21:18:27.545310 32503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "e92aa07aaee74cd3b83a6655cb7859d2"
format_stamp: "Formatted at 2026-09-24 21:18:27 on dist-test-slave-cxpd"
server_key: "e1bfd7ac55e9008939ecdb59804875f8"
server_key_iv: "0719e8fa74d0f9c1857a18aa9abbf8e0"
server_key_version: "kuduclusterkey@0"
I20260924 21:18:27.545424 32503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:18:27.571743 32503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:18:27.574143 32503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:18:27.574285 32503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:18:27.574497 32503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:18:27.574828 32503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:18:27.574887 32503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:27.574918 32503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:18:27.574951 32503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:18:27.582108 32503 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:46333
I20260924 21:18:27.582525 32503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:18:27.586478 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 32503
I20260924 21:18:27.586614 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:18:27.586833 20943 external_mini_cluster.cc:1428] Setting key cb95fd867fc32aa313c6f173aa625fd2
I20260924 21:18:27.588975 32631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:46333 every 8 connection(s)
Sep 24 21:18:27 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284707, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:18:27.600521 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.589635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:52157 (local address 127.20.115.254:43809)
0924 21:18:27.589680 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.589682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.589982 (+ 300us) server_negotiation.cc:407] Connection header received
0924 21:18:27.591030 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.591031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.591044 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.591075 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:18:27.591910 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.592729 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.594282 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.594585 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.594625 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.597265 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:18:27.597267 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:18:27.597267 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:18:27.597275 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:18:27.598915 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:27.599463 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:27.599464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:27.599464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:27.599523 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:27.599816 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:27.599816 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:27.599817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:27.599958 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:18:27.600025 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.600345 (+ 320us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.600424 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:18:27.600701 32634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.589654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43809 (local address 127.20.115.195:52157)
0924 21:18:27.589843 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 21:18:27.589894 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0924 21:18:27.590531 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:18:27.591275 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:18:27.591280 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:18:27.591530 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:18:27.591808 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.591814 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.592888 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.592889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.594181 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.594183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.594712 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.594713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.594851 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.595501 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:18:27.595512 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:18:27.597162 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:18:27.599008 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:18:27.599015 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:18:27.599017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:18:27.599385 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:18:27.599600 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:18:27.599601 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:18:27.599601 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:18:27.599739 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:18:27.600029 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:18:27.600034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:18:27.600193 (+ 159us) client_negotiation.cc:769] Sending connection context
0924 21:18:27.600310 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 21:18:27.600459 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":98,"threads_started":1}
I20260924 21:18:27.601305 32632 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43809
I20260924 21:18:27.601404 32632 heartbeater.cc:464] Registering TS with master...
I20260924 21:18:27.601586 32632 heartbeater.cc:511] Master 127.20.115.254:43809 requested a full tablet report, sending...
I20260924 21:18:27.601953 32148 ts_manager.cc:194] Registered new tserver with Master: e92aa07aaee74cd3b83a6655cb7859d2 (127.20.115.195:46333)
I20260924 21:18:27.602428 32148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:52157
I20260924 21:18:27.611654 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:18:27.622396 32362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.613406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58750 (local address 127.20.115.254:43809)
0924 21:18:27.613448 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.613450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.613553 (+ 103us) server_negotiation.cc:407] Connection header received
0924 21:18:27.613646 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.613646 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.613656 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.613685 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:18:27.615047 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.615811 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.617078 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.617372 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.617410 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.618853 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:18:27.618854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:18:27.618855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:18:27.618862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:18:27.620816 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:27.621572 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:27.621574 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:27.621574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:27.621629 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:18:27.621811 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:18:27.621812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:18:27.621812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:18:27.621944 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:18:27.622020 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.622224 (+ 204us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.622300 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Sep 24 21:18:27 dist-test-slave-cxpd krb5kdc[31672](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284707, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:18:27.629030 32148 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:18:27.631210 32148 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:18:27.639940 32644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.634996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:33521 (local address 127.0.0.1:47790)
0924 21:18:27.635177 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0924 21:18:27.635471 (+ 294us) client_negotiation.cc:174] Beginning negotiation
0924 21:18:27.635537 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:18:27.636159 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:18:27.636162 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:18:27.636165 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:18:27.636232 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.636234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.637445 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.637446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.638708 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.638711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.639658 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.639659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.639741 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.639758 (+ 17us) client_negotiation.cc:769] Sending connection context
0924 21:18:27.639805 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 21:18:27.639856 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260924 21:18:27.640297 32649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.635810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47790 (local address 127.20.115.194:33521)
0924 21:18:27.635973 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.635978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.635988 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:18:27.636028 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.636030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.636046 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.636125 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:18:27.636320 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.637337 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.638838 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.639541 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.640086 (+ 545us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.640155 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.640185 (+ 30us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.640222 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260924 21:18:27.641074 32432 tablet_service.cc:1467] Processing CreateTablet for tablet b3696ddbc6df483aa64dd14521f681ad (DEFAULT_TABLE table=test-table [id=cd2dffcc5cee4bfd819214a67b5e19a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:18:27.641469 32432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3696ddbc6df483aa64dd14521f681ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:18:27.642019 32647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.635457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:36329 (local address 127.0.0.1:46142)
0924 21:18:27.635735 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0924 21:18:27.635746 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:18:27.635802 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:18:27.637097 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:18:27.637098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:18:27.637100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:18:27.637140 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.637141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.638671 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.638672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.639863 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.639865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.641791 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.641792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.641860 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.641865 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:18:27.641895 (+ 30us) client_negotiation.cc:240] Negotiation successful
0924 21:18:27.641947 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":28,"thread_start_us":62,"threads_started":1}
I20260924 21:18:27.642571 32648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.635581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.20.115.193:36329)
0924 21:18:27.636904 (+ 1323us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.636909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.636919 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:18:27.636960 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.636961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.636978 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.637445 (+ 467us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:18:27.637507 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.638572 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.640771 (+ 2199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.641693 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.642362 (+ 669us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.642428 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.642455 (+ 27us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.642495 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1291,"thread_start_us":56,"threads_started":1}
I20260924 21:18:27.643329 32294 tablet_service.cc:1467] Processing CreateTablet for tablet b3696ddbc6df483aa64dd14521f681ad (DEFAULT_TABLE table=test-table [id=cd2dffcc5cee4bfd819214a67b5e19a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:18:27.643690 32294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3696ddbc6df483aa64dd14521f681ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:18:27.645226 32650 tablet_bootstrap.cc:492] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: Bootstrap starting.
I20260924 21:18:27.646270 32650 tablet_bootstrap.cc:654] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: Neither blocks nor log segments found. Creating new log.
I20260924 21:18:27.646976 32650 log.cc:824] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: Log is configured to *not* fsync() on all Append() calls
I20260924 21:18:27.647382 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.635263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:46333 (local address 127.0.0.1:55488)
0924 21:18:27.635645 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0924 21:18:27.635653 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:18:27.635687 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:18:27.643586 (+ 7899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:18:27.643587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:18:27.643590 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:18:27.643631 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.643633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.645102 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.645103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.646085 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.646088 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.647141 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.647142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.647199 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.647203 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:18:27.647259 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 21:18:27.647297 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":74,"threads_started":1}
I20260924 21:18:27.647681 32646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.635372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55488 (local address 127.20.115.195:46333)
0924 21:18:27.643063 (+ 7691us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.643068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.643078 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:18:27.643403 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.643407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.643427 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.643708 (+ 281us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:18:27.643835 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.645037 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.646200 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.646961 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.647047 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.647102 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.647547 (+ 445us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.647605 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7651,"thread_start_us":77,"threads_started":1}
I20260924 21:18:27.647956 32650 tablet_bootstrap.cc:492] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: No bootstrap required, opened a new log
I20260924 21:18:27.648012 32650 ts_tablet_manager.cc:1397] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 21:18:27.648272 32553 tablet_service.cc:1467] Processing CreateTablet for tablet b3696ddbc6df483aa64dd14521f681ad (DEFAULT_TABLE table=test-table [id=cd2dffcc5cee4bfd819214a67b5e19a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:18:27.648604 32553 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b3696ddbc6df483aa64dd14521f681ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:18:27.649865 32650 raft_consensus.cc:348] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } }
I20260924 21:18:27.650033 32650 raft_consensus.cc:374] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:18:27.650064 32650 raft_consensus.cc:729] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeadd2b5ff648e99e576a9c2ec364a4, State: Initialized, Role: FOLLOWER
I20260924 21:18:27.650173 32650 consensus_queue.cc:260] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } }
I20260924 21:18:27.650403 32650 ts_tablet_manager.cc:1428] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:18:27.650606 32498 heartbeater.cc:503] Master 127.20.115.254:43809 was elected leader, sending a full tablet report...
I20260924 21:18:27.652977 32651 tablet_bootstrap.cc:492] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689: Bootstrap starting.
I20260924 21:18:27.653751 32651 tablet_bootstrap.cc:654] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689: Neither blocks nor log segments found. Creating new log.
I20260924 21:18:27.654137 32651 log.cc:824] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689: Log is configured to *not* fsync() on all Append() calls
I20260924 21:18:27.654516 32654 tablet_bootstrap.cc:492] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: Bootstrap starting.
I20260924 21:18:27.655017 32651 tablet_bootstrap.cc:492] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689: No bootstrap required, opened a new log
I20260924 21:18:27.655084 32651 ts_tablet_manager.cc:1397] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:18:27.655629 32654 tablet_bootstrap.cc:654] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: Neither blocks nor log segments found. Creating new log.
I20260924 21:18:27.655934 32654 log.cc:824] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: Log is configured to *not* fsync() on all Append() calls
I20260924 21:18:27.656824 32654 tablet_bootstrap.cc:492] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: No bootstrap required, opened a new log
I20260924 21:18:27.656891 32654 ts_tablet_manager.cc:1397] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:18:27.657528 32651 raft_consensus.cc:348] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } }
I20260924 21:18:27.657686 32651 raft_consensus.cc:374] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:18:27.657716 32651 raft_consensus.cc:729] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c369b75efded4c30bb97fc9e7d02f689, State: Initialized, Role: FOLLOWER
I20260924 21:18:27.658226 32651 consensus_queue.cc:260] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } }
I20260924 21:18:27.658511 32651 ts_tablet_manager.cc:1428] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:18:27.658620 32360 heartbeater.cc:503] Master 127.20.115.254:43809 was elected leader, sending a full tablet report...
I20260924 21:18:27.658938 32654 raft_consensus.cc:348] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } }
I20260924 21:18:27.659075 32654 raft_consensus.cc:374] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:18:27.659097 32654 raft_consensus.cc:729] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e92aa07aaee74cd3b83a6655cb7859d2, State: Initialized, Role: FOLLOWER
I20260924 21:18:27.659221 32654 consensus_queue.cc:260] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } }
I20260924 21:18:27.659471 32654 ts_tablet_manager.cc:1428] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260924 21:18:27.659510 32632 heartbeater.cc:503] Master 127.20.115.254:43809 was elected leader, sending a full tablet report...
I20260924 21:18:27.754447 32657 raft_consensus.cc:482] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:18:27.754609 32657 raft_consensus.cc:504] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } }
I20260924 21:18:27.755029 32657 leader_election.cc:288] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeadd2b5ff648e99e576a9c2ec364a4 (127.20.115.194:33521), e92aa07aaee74cd3b83a6655cb7859d2 (127.20.115.195:46333)
I20260924 21:18:27.759243 32649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.755338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:33623 (local address 127.20.115.194:33521)
0924 21:18:27.755385 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.755387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.755765 (+ 378us) server_negotiation.cc:407] Connection header received
0924 21:18:27.755808 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.755809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.755825 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.755904 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:18:27.756158 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.756908 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.758403 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.758888 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.758940 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.758949 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.759141 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.759177 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:18:27.759348 32646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.755531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:51383 (local address 127.20.115.195:46333)
0924 21:18:27.755581 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.755584 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.755905 (+ 321us) server_negotiation.cc:407] Connection header received
0924 21:18:27.756009 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.756010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.756020 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.756049 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:18:27.756279 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.757021 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.758426 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.758952 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.759001 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.759009 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.759238 (+ 229us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.759281 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:18:27.759475 32659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.755231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:33521 (local address 127.20.115.193:33623)
0924 21:18:27.755672 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0924 21:18:27.755685 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:18:27.755751 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:18:27.755981 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:18:27.755982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:18:27.755985 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:18:27.756074 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.756076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.757033 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.757034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.758300 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.758303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.759005 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.759006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.759068 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.759072 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:18:27.759350 (+ 278us) client_negotiation.cc:240] Negotiation successful
0924 21:18:27.759391 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"mutex_wait_us":45,"thread_start_us":104,"threads_started":1}
I20260924 21:18:27.759778 32449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3696ddbc6df483aa64dd14521f681ad" candidate_uuid: "c369b75efded4c30bb97fc9e7d02f689" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" is_pre_election: true
I20260924 21:18:27.759944 32449 raft_consensus.cc:2454] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c369b75efded4c30bb97fc9e7d02f689 in term 0.
I20260924 21:18:27.760092 32660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.755455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:46333 (local address 127.20.115.193:51383)
0924 21:18:27.755855 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0924 21:18:27.755863 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:18:27.755937 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:18:27.756130 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:18:27.756131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:18:27.756132 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:18:27.756206 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.756207 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.757147 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.757147 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.758317 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:18:27.758321 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.759080 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.759081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:18:27.759150 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.759154 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:18:27.759829 (+ 675us) client_negotiation.cc:240] Negotiation successful
0924 21:18:27.759941 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":2,"thread_start_us":75,"threads_started":1}
I20260924 21:18:27.760154 32248 leader_election.cc:302] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8eeadd2b5ff648e99e576a9c2ec364a4, c369b75efded4c30bb97fc9e7d02f689; no voters:
I20260924 21:18:27.760370 32657 raft_consensus.cc:2781] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:18:27.760433 32657 raft_consensus.cc:482] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:18:27.760463 32657 raft_consensus.cc:3037] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:18:27.760594 32586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3696ddbc6df483aa64dd14521f681ad" candidate_uuid: "c369b75efded4c30bb97fc9e7d02f689" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" is_pre_election: true
I20260924 21:18:27.760720 32586 raft_consensus.cc:2454] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c369b75efded4c30bb97fc9e7d02f689 in term 0.
I20260924 21:18:27.761413 32657 raft_consensus.cc:504] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } }
I20260924 21:18:27.761544 32657 leader_election.cc:288] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [CANDIDATE]: Term 1 election: Requested vote from peers 8eeadd2b5ff648e99e576a9c2ec364a4 (127.20.115.194:33521), e92aa07aaee74cd3b83a6655cb7859d2 (127.20.115.195:46333)
I20260924 21:18:27.761752 32586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3696ddbc6df483aa64dd14521f681ad" candidate_uuid: "c369b75efded4c30bb97fc9e7d02f689" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e92aa07aaee74cd3b83a6655cb7859d2"
I20260924 21:18:27.761826 32586 raft_consensus.cc:3037] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:18:27.761900 32449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b3696ddbc6df483aa64dd14521f681ad" candidate_uuid: "c369b75efded4c30bb97fc9e7d02f689" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4"
I20260924 21:18:27.761976 32449 raft_consensus.cc:3037] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:18:27.762616 32586 raft_consensus.cc:2454] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c369b75efded4c30bb97fc9e7d02f689 in term 1.
I20260924 21:18:27.762768 32248 leader_election.cc:302] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c369b75efded4c30bb97fc9e7d02f689, e92aa07aaee74cd3b83a6655cb7859d2; no voters:
I20260924 21:18:27.762813 32449 raft_consensus.cc:2454] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c369b75efded4c30bb97fc9e7d02f689 in term 1.
I20260924 21:18:27.762982 32657 raft_consensus.cc:2781] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:18:27.763152 32657 raft_consensus.cc:686] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [term 1 LEADER]: Becoming Leader. State: Replica: c369b75efded4c30bb97fc9e7d02f689, State: Running, Role: LEADER
I20260924 21:18:27.763248 32657 consensus_queue.cc:237] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } }
I20260924 21:18:27.763989 32148 catalog_manager.cc:5495] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 reported cstate change: term changed from 0 to 1, leader changed from <none> to c369b75efded4c30bb97fc9e7d02f689 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "c369b75efded4c30bb97fc9e7d02f689" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c369b75efded4c30bb97fc9e7d02f689" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 } health_report { overall_health: UNKNOWN } } }
I20260924 21:18:27.786674 32648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.783515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46146 (local address 127.20.115.193:36329)
0924 21:18:27.783554 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:18:27.783557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:18:27.783564 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:18:27.783603 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:18:27.783604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:18:27.783625 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:18:27.783697 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:18:27.783875 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.784597 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.785709 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:18:27.785994 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:18:27.786060 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:18:27.786269 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:18:27.786398 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:18:27.786480 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:18:27.786553 (+ 73us) server_negotiation.cc:299] Negotiation successful
0924 21:18:27.786575 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:18:27.788991 32586 raft_consensus.cc:1261] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [term 1 FOLLOWER]: Refusing update from remote peer c369b75efded4c30bb97fc9e7d02f689: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:18:27.789011 32449 raft_consensus.cc:1261] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [term 1 FOLLOWER]: Refusing update from remote peer c369b75efded4c30bb97fc9e7d02f689: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:18:27.789335 32661 consensus_queue.cc:1035] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e92aa07aaee74cd3b83a6655cb7859d2" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 46333 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:18:27.789466 32657 consensus_queue.cc:1035] T b3696ddbc6df483aa64dd14521f681ad P c369b75efded4c30bb97fc9e7d02f689 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeadd2b5ff648e99e576a9c2ec364a4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 33521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:18:27.792168 32664 mvcc.cc:205] Tried to move back new op lower bound from 7333006163100090368 to 7333006162998788096. Current Snapshot: MvccSnapshot[applied={T|T < 7333006163100090368}]
I20260924 21:18:27.794252 32666 mvcc.cc:205] Tried to move back new op lower bound from 7333006163100090368 to 7333006162998788096. Current Snapshot: MvccSnapshot[applied={T|T < 7333006163100090368}]
I20260924 21:18:27.794436 32665 mvcc.cc:205] Tried to move back new op lower bound from 7333006163100090368 to 7333006162998788096. Current Snapshot: MvccSnapshot[applied={T|T < 7333006163100090368}]
I20260924 21:18:27.794950 32148 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:18:27.795069 32148 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:18:27.795907 32148 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00073874a08b42c1bdc04b3c1969ae3f: Sending DeleteTablet for 3 replicas of tablet b3696ddbc6df483aa64dd14521f681ad
I20260924 21:18:27.796408 32294 tablet_service.cc:1514] Processing DeleteTablet for tablet b3696ddbc6df483aa64dd14521f681ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:18:27 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:46142
I20260924 21:18:27.796473 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 32229
I20260924 21:18:27.797026 32432 tablet_service.cc:1514] Processing DeleteTablet for tablet b3696ddbc6df483aa64dd14521f681ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:18:27 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:47790
I20260924 21:18:27.797230 32553 tablet_service.cc:1514] Processing DeleteTablet for tablet b3696ddbc6df483aa64dd14521f681ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:18:27 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:55488
I20260924 21:18:27.797786 32674 tablet_replica.cc:330] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: stopping tablet replica
I20260924 21:18:27.797880 32674 raft_consensus.cc:2229] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:18:27.797992 32674 raft_consensus.cc:2258] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:18:27.798434 32674 ts_tablet_manager.cc:1905] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:18:27.800171 32674 ts_tablet_manager.cc:1918] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:18:27.800259 32674 log.cc:1196] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/b3696ddbc6df483aa64dd14521f681ad
I20260924 21:18:27.800545 32674 ts_tablet_manager.cc:1939] T b3696ddbc6df483aa64dd14521f681ad P 8eeadd2b5ff648e99e576a9c2ec364a4: Deleting consensus metadata
I20260924 21:18:27.801151 32133 catalog_manager.cc:4841] TS 8eeadd2b5ff648e99e576a9c2ec364a4 (127.20.115.194:33521): tablet b3696ddbc6df483aa64dd14521f681ad (table test-table [id=cd2dffcc5cee4bfd819214a67b5e19a0]) successfully deleted
I20260924 21:18:27.801468 32675 tablet_replica.cc:330] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: stopping tablet replica
I20260924 21:18:27.801599 32675 raft_consensus.cc:2229] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:18:27.801703 32675 raft_consensus.cc:2258] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:18:27.802179 32675 ts_tablet_manager.cc:1905] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:18:27.804180 32675 ts_tablet_manager.cc:1918] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:18:27.804304 32675 log.cc:1196] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/b3696ddbc6df483aa64dd14521f681ad
I20260924 21:18:27.804946 32675 ts_tablet_manager.cc:1939] T b3696ddbc6df483aa64dd14521f681ad P e92aa07aaee74cd3b83a6655cb7859d2: Deleting consensus metadata
I20260924 21:18:27.805418 32133 catalog_manager.cc:4841] TS e92aa07aaee74cd3b83a6655cb7859d2 (127.20.115.195:46333): tablet b3696ddbc6df483aa64dd14521f681ad (table test-table [id=cd2dffcc5cee4bfd819214a67b5e19a0]) successfully deleted
W20260924 21:18:27.805783 32134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:36329 (error 108)
I20260924 21:18:27.806165 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 32368
I20260924 21:18:27.807065 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:18:27.806944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:36329 (local address 127.0.0.1:46154)
0924 21:18:27.806991 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:18:27.807024 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:36329: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 21:18:27.807178 32134 catalog_manager.cc:4584] TS c369b75efded4c30bb97fc9e7d02f689 (127.20.115.193:36329): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b3696ddbc6df483aa64dd14521f681ad: Network error: Client connection negotiation failed: client connection to 127.20.115.193:36329: connect: Connection refused (error 111)
I20260924 21:18:27.812323 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 32503
I20260924 21:18:27.817296 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 32116
I20260924 21:18:27.823927 20943 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 21:18:28.158388 20943 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 21:18:28.158521 20943 mini_ranger.cc:66] Stopping Ranger...
I20260924 21:18:28.503252 20943 mini_ranger.cc:68] Stopped Ranger
2026-09-24 21:18:28.503 UTC [31702] LOG: received smart shutdown request
2026-09-24 21:18:28.506 UTC [31702] LOG: background worker "logical replication launcher" (PID 31711) exited with exit code 1
2026-09-24 21:18:28.507 UTC [31706] LOG: shutting down
2026-09-24 21:18:28.693 UTC [31702] LOG: database system is shut down
2026-09-24T21:18:28Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46067 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:18:28 dist-test-slave-cxpd krb5kdc[32685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:18:28 dist-test-slave-cxpd krb5kdc[32685](info): set up 2 sockets
Sep 24 21:18:28 dist-test-slave-cxpd krb5kdc[32685](info): commencing operation
krb5kdc: starting...
W20260924 21:18:29.840739 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:18:29 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:18:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:18:29Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/rangeradmin_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/rangeradmin_127.20.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/rangerlookup_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/rangerlookup_127.20.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/HTTP_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/HTTP_127.20.115.212@KRBTEST.COM.keytab.
I20260924 21:18:30.012205 20943 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskNZGLg7/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/postgres -l logfile start
2026-09-24 21:18:32.522 UTC [32715] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-24 21:18:32.522 UTC [32715] LOG: listening on IPv4 address "127.20.115.212", port 51871
2026-09-24 21:18:32.526 UTC [32715] LOG: listening on Unix socket "/tmp/.s.PGSQL.51871"
2026-09-24 21:18:32.547 UTC [32718] LOG: database system was shut down at 2026-09-24 21:18:31 UTC
2026-09-24 21:18:32.551 UTC [32715] LOG: database system is ready to accept connections
I20260924 21:18:33.496551 20943 mini_postgres.cc:96] Postgres bound to 51871
2026-09-24 21:18:33.502 UTC [32726] FATAL: database "slave" does not exist
127.20.115.212:51871 - accepting connections
I20260924 21:18:33.503129 20943 mini_ranger.cc:161] Starting Ranger...
I20260924 21:18:33.517186 20943 mini_ranger.cc:84] Created miniranger Postgres user
I20260924 21:18:33.998618 20943 mini_ranger.cc:87] Created ranger Postgres database
I20260924 21:18:33.998772 20943 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-admin
2026-09-24 21:18:34,275 [I] DB FLAVOR :POSTGRES
2026-09-24 21:18:34,276 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 21:18:34,276 [I] Checking connection..
2026-09-24 21:18:34,276 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-24 21:18:34,534 [I] Checking connection passed.
2026-09-24 21:18:34,534 [I] --------- Verifying version history table ---------
2026-09-24 21:18:34,534 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:18:34,795 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 21:18:34,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:18:35,020 [I] Table x_db_version_h does not exist in database ranger
2026-09-24 21:18:35,020 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-24 21:18:35,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-24 21:18:35.268 UTC [322] WARNING: there is no transaction in progress
2026-09-24 21:18:35,275 [I] create_dbversion_catalog.sql file imported successfully
2026-09-24 21:18:35,275 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-24 21:18:35,513 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-24 21:18:35,514 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 21:18:35,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-24 21:18:35,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-24 21:18:35,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cxpd.c.gcp-upstream.internal','N') ;"
2026-09-24 21:18:36,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 21:18:36,496 [I] Table x_portal_user does not exist in database ranger
2026-09-24 21:18:36,497 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 21:18:36,748 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-24 21:18:36,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 21:18:36,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 21:18:37,255 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-24 21:18:37,256 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-24 21:18:38.277 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.294 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.311 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.327 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.346 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.363 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.418 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.424 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.439 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.449 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.459 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.472 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.482 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.488 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.495 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:38.502 UTC [430] WARNING: there is no transaction in progress
2026-09-24 21:18:39,166 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-24 21:18:39,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-24 21:18:39,384 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-24 21:18:39,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-24 21:18:39,619 [I] Table x_portal_user already exists in database 'ranger'
2026-09-24 21:18:39,619 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-24 21:18:39,871 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-24 21:18:39,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-24 21:18:40,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-24 21:18:40,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cxpd.c.gcp-upstream.internal';"
2026-09-24 21:18:40,594 [I] CORE_DB_SCHEMA import status has been updated
2026-09-24 21:18:40,594 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-24 21:18:40,837 [I] DB_PATCHES have already been applied
I20260924 21:18:40.842978 20943 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-admin:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskNZGLg7/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260924 21:18:40.843039 20943 mini_ranger.cc:193] Using host: 127.20.115.212
I20260924 21:18:40.845968 20943 mini_ranger.cc:227] Ranger admin URL: http://127.20.115.212:38175
Sep 24, 2026 9:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 9:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 9:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-admin/ews/webapp
Sep 24, 2026 9:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 24, 2026 9:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 24, 2026 9:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-admin/ews/webapp].
Sep 24, 2026 9:18:41 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260924 21:18:41.975950 20943 mini_ranger.cc:160] Time spent starting Ranger: real 8.473s user 0.000s sys 0.008s
Sep 24, 2026 9:19:01 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 24, 2026 9:19:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 24, 2026 9:19:01 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 24, 2026 9:19:01 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 24, 2026 9:19:02 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260924 21:19:02.901541 20943 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.115.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.115.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/rangerkms_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.115.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/rangerkms_127.20.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.115.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.115.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.115.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/HTTP_127.20.115.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.115.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/HTTP_127.20.115.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 24 21:19:02 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284742, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260924 21:19:02.989236 20943 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260924 21:19:02.998574 20943 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260924 21:19:03.931656 20943 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260924 21:19:03.931741 20943 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-kms
I20260924 21:19:03.931764 20943 mini_ranger_kms.cc:227] Using postgres at 127.20.115.212:51871
2026-09-24 21:19:04,088 [I] DB FLAVOR :POSTGRES
2026-09-24 21:19:04,088 [I] --------- Verifying Ranger DB connection ---------
2026-09-24 21:19:04,088 [I] Checking connection
2026-09-24 21:19:04,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-24 21:19:04,320 [I] connection success
2026-09-24 21:19:04,321 [I] --------- Verifying Ranger DB tables ---------
2026-09-24 21:19:04,321 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-24 21:19:04,321 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-24 21:19:04,548 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-24 21:19:04,548 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-24 21:19:04,548 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-24 21:19:04,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.115.212:51871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-24 21:19:04,824 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260924 21:19:05.334440 20943 mini_ranger_kms.cc:310] Created kms service
I20260924 21:19:05.547199 20943 mini_ranger_kms.cc:326] Created kudu user
I20260924 21:19:05.589200 20943 mini_ranger_kms.cc:343] Created rangerkms user
Sep 24 21:19:05 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284742, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:05.976903 20943 mini_ranger_kms.cc:384] Added ranger policy
I20260924 21:19:05.977021 20943 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-kms:/tmp/dist-test-taskNZGLg7/build/release/bin/postgresql.jar:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskNZGLg7/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260924 21:19:05.977043 20943 mini_ranger_kms.cc:240] Using host: 127.20.115.212
I20260924 21:19:05.980372 20943 mini_ranger_kms.cc:276] Ranger KMS PID: 610
I20260924 21:19:05.980458 20943 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.115.212:48967
Sep 24, 2026 9:19:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 24, 2026 9:19:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 24, 2026 9:19:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 24, 2026 9:19:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 24, 2026 9:19:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-kms/ews/webapp].
Sep 24, 2026 9:19:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260924 21:19:08.223398 20943 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.234s user 0.022s sys 0.118s
I20260924 21:19:08.223505 20943 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260924 21:19:08.223556 20943 mini_ranger_kms.cc:401] 127.20.115.212:48967/kms/v1/keys
Sep 24 21:19:08 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284748, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.115.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 24 21:19:08 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284748, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.115.212@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
Sep 24 21:19:09 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284748, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.115.212@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
Sep 24 21:19:11 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:19:11.380645 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45565
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:45565
--ranger_config_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:48967/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:11.459410 661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:11.459612 661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:11.459633 661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:11.460834 661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:11.460875 661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:11.460888 661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:11.460901 661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:11.460942 661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:11.462467 661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.115.212:48967/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:45565
--ranger_config_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:11.462694 661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:11.462919 661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:11.467710 669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:11.467768 666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:11.467770 667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:11.467933 661 server_base.cc:1034] running on GCE node
I20260924 21:19:11.468207 661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:11.468441 661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:11.469574 661 hybrid_clock.cc:648] HybridClock initialized: now 1790284751469556 us; error 39 us; skew 500 ppm
Sep 24 21:19:11 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284751, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:11.471944 661 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:19:11.472399 661 webserver.cc:460] Webserver started at http://127.20.115.254:33881/ using document root <none> and password file <none>
I20260924 21:19:11.472616 661 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:11.472674 661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:11.472851 661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:19:11 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284751, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:11.523985 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "68385bf56e8f44babe6219a605322b1a"
format_stamp: "Formatted at 2026-09-24 21:19:11 on dist-test-slave-cxpd"
server_key: "6241e0e626a1ca0a0d402a08f511bbf7"
server_key_iv: "f48a6b583b64dcd06ee16d43164387ad"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:11.524369 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "68385bf56e8f44babe6219a605322b1a"
format_stamp: "Formatted at 2026-09-24 21:19:11 on dist-test-slave-cxpd"
server_key: "6241e0e626a1ca0a0d402a08f511bbf7"
server_key_iv: "f48a6b583b64dcd06ee16d43164387ad"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:11.525928 661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260924 21:19:11.537060 676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:11.537233 661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:11.537334 661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "68385bf56e8f44babe6219a605322b1a"
format_stamp: "Formatted at 2026-09-24 21:19:11 on dist-test-slave-cxpd"
server_key: "6241e0e626a1ca0a0d402a08f511bbf7"
server_key_iv: "f48a6b583b64dcd06ee16d43164387ad"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:11.537446 661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:11.554569 661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:11.556555 661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:11.556720 661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:11.560493 661 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:45565
I20260924 21:19:11.560590 728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:45565 every 8 connection(s)
I20260924 21:19:11.560986 661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:19:11.561735 729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:11.564803 729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Bootstrap starting.
I20260924 21:19:11.565491 729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:11.565797 729 log.cc:824] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:11.566381 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 661
I20260924 21:19:11.566491 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:19:11.566542 729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: No bootstrap required, opened a new log
I20260924 21:19:11.566681 20943 external_mini_cluster.cc:1428] Setting key 486bcacc0c8be020276a0022df3b91dd
I20260924 21:19:11.569435 729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } }
I20260924 21:19:11.569618 729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:11.569657 729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68385bf56e8f44babe6219a605322b1a, State: Initialized, Role: FOLLOWER
I20260924 21:19:11.569823 729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } }
I20260924 21:19:11.569905 729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:19:11.569933 729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:19:11.569998 729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:11.570663 729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } }
I20260924 21:19:11.570767 729 leader_election.cc:302] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68385bf56e8f44babe6219a605322b1a; no voters:
I20260924 21:19:11.570959 729 leader_election.cc:288] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:19:11.571009 734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:19:11.571225 734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 1 LEADER]: Becoming Leader. State: Replica: 68385bf56e8f44babe6219a605322b1a, State: Running, Role: LEADER
I20260924 21:19:11.571346 729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:19:11.571377 734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } }
I20260924 21:19:11.572130 734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "68385bf56e8f44babe6219a605322b1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } } }
I20260924 21:19:11.572278 734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:11.572458 734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68385bf56e8f44babe6219a605322b1a. Latest consensus state: current_term: 1 leader_uuid: "68385bf56e8f44babe6219a605322b1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } } }
I20260924 21:19:11.572567 734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:11.572722 743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:19:11.573092 729 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 21:19:11.573346 743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 21:19:11 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:11.580878 733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:11.567209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.20.115.254:45565)
0924 21:19:11.567416 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:11.567425 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:11.567452 (+ 27us) server_negotiation.cc:407] Connection header received
0924 21:19:11.567864 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:11.567870 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:11.567913 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:11.568061 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:11.568666 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:11.569607 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:11.570512 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:11.571081 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:11.571490 (+ 409us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:11.575524 (+ 4034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:11.575584 (+ 60us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:11.575584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:11.575686 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:11.578811 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:11.579212 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:11.579214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:11.579215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:11.579302 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:11.579519 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:11.579520 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:11.579521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:11.579820 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:11.580387 (+ 567us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:11.580465 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 21:19:11.580629 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":77,"threads_started":1}
I20260924 21:19:11.586304 743 catalog_manager.cc:1347] Generated new cluster ID: bd1651d273b34afd8b3f07102fe8fa9f
I20260924 21:19:11.586360 743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:19:11.596381 743 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:19:11.597172 743 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:19:11.604928 743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Generated new TSK 0
I20260924 21:19:11.605165 743 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:19:12.268273 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:48967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45565
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:12.385475 774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:12.385746 774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:12.385780 774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:12.387418 774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:12.387481 774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:12.387526 774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:19:12.389611 774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.115.212:48967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:12.389853 774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:12.390130 774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:12.395427 780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:12.395481 779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:12.395797 782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:12.396358 774 server_base.cc:1034] running on GCE node
I20260924 21:19:12.396546 774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:12.396811 774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:12.398000 774 hybrid_clock.cc:648] HybridClock initialized: now 1790284752397983 us; error 32 us; skew 500 ppm
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:12.400722 774 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:19:12.401198 774 webserver.cc:460] Webserver started at http://127.20.115.193:44037/ using document root <none> and password file <none>
I20260924 21:19:12.401438 774 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:12.401506 774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:12.401636 774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:12.416028 774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "1feb7a1a2ebc47ad8e1facab9a255b03"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "229f16a16ac95e2a85a5385392e3f1f0"
server_key_iv: "68fec6df8867f9ad0c87f48935e5cd8d"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:12.416486 774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "1feb7a1a2ebc47ad8e1facab9a255b03"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "229f16a16ac95e2a85a5385392e3f1f0"
server_key_iv: "68fec6df8867f9ad0c87f48935e5cd8d"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:12.418422 774 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260924 21:19:12.430990 789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:12.431205 774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:12.431304 774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "1feb7a1a2ebc47ad8e1facab9a255b03"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "229f16a16ac95e2a85a5385392e3f1f0"
server_key_iv: "68fec6df8867f9ad0c87f48935e5cd8d"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:12.431399 774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:12.444620 774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:12.447060 774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:12.447255 774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:12.447554 774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:12.447950 774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:12.448004 774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:12.448036 774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:12.448055 774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:12.454839 774 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:35485
I20260924 21:19:12.455267 774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:19:12.460979 904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:35485 every 8 connection(s)
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:12.464365 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 774
I20260924 21:19:12.464462 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:19:12.464651 20943 external_mini_cluster.cc:1428] Setting key 08b53c8b40e37400af8f1279b8c9dbda
I20260924 21:19:12.467299 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:12.456074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:39541 (local address 127.20.115.254:45565)
0924 21:19:12.456209 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:12.456211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:12.456877 (+ 666us) server_negotiation.cc:407] Connection header received
0924 21:19:12.457570 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:12.457572 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:12.457585 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:12.457670 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:12.458376 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.459186 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.460679 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.461008 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.461082 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:12.463614 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:12.463615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:12.463616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:12.463624 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:12.465604 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:12.466100 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:12.466101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:12.466101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:12.466168 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:12.466469 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:12.466470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:12.466471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:12.466631 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:12.466682 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:12.467068 (+ 386us) server_negotiation.cc:299] Negotiation successful
0924 21:19:12.467170 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260924 21:19:12.467397 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:12.456038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45565 (local address 127.20.115.193:39541)
0924 21:19:12.456715 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:12.456762 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:12.457445 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:12.457740 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:12.457744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:12.457997 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:12.458261 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:12.458267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.459289 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.459290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:12.460580 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:12.460583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.461165 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.461166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:12.461313 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:12.461880 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:12.461891 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:12.463506 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:12.465700 (+ 2194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:12.465706 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:12.465709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:12.466002 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:12.466249 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:12.466250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:12.466251 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:12.466386 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:12.466734 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:12.466739 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:12.466899 (+ 160us) client_negotiation.cc:769] Sending connection context
0924 21:19:12.467016 (+ 117us) client_negotiation.cc:240] Negotiation successful
0924 21:19:12.467169 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"thread_start_us":580,"threads_started":1}
I20260924 21:19:12.468212 905 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45565
I20260924 21:19:12.468313 905 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:12.468500 905 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
I20260924 21:19:12.469167 693 ts_manager.cc:194] Registered new tserver with Master: 1feb7a1a2ebc47ad8e1facab9a255b03 (127.20.115.193:35485)
I20260924 21:19:12.470105 693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:39541
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:19:12.491519 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:48967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45565
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:12.594039 911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:12.594265 911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:12.594300 911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:12.595431 911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:12.595498 911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:12.595542 911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:19:12.597050 911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.115.212:48967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:12.597293 911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:12.597544 911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:12.602866 919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:12.602856 921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:12.603022 918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:12.603178 911 server_base.cc:1034] running on GCE node
I20260924 21:19:12.603547 911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:12.603822 911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:12.604954 911 hybrid_clock.cc:648] HybridClock initialized: now 1790284752604938 us; error 33 us; skew 500 ppm
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:12.607551 911 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:19:12.608040 911 webserver.cc:460] Webserver started at http://127.20.115.194:37495/ using document root <none> and password file <none>
I20260924 21:19:12.608286 911 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:12.608353 911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:12.608479 911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:12.621140 911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "9558cfdfa1884161a631d51a67557350"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "6849de5db38f9c7dd8bb0668049828a6"
server_key_iv: "81d29b5589df082af3315d459f9330aa"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:12.621651 911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "9558cfdfa1884161a631d51a67557350"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "6849de5db38f9c7dd8bb0668049828a6"
server_key_iv: "81d29b5589df082af3315d459f9330aa"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:12.624464 911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:12.658048 928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:12.658309 911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:19:12.658402 911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "9558cfdfa1884161a631d51a67557350"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "6849de5db38f9c7dd8bb0668049828a6"
server_key_iv: "81d29b5589df082af3315d459f9330aa"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:12.658495 911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:12.685820 911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:12.688042 911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:12.688243 911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:12.688519 911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:12.688954 911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:12.689011 911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:12.689085 911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:12.689116 911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:12.696360 911 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:35989
I20260924 21:19:12.696918 911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:19:12.697443 1041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:35989 every 8 connection(s)
I20260924 21:19:12.702323 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 911
I20260924 21:19:12.702422 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:19:12.702646 20943 external_mini_cluster.cc:1428] Setting key 4263f47799a5b657f2912c422eb2028c
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:12.708526 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:12.698363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:43427 (local address 127.20.115.254:45565)
0924 21:19:12.698410 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:12.698413 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:12.698726 (+ 313us) server_negotiation.cc:407] Connection header received
0924 21:19:12.699437 (+ 711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:12.699439 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:12.699456 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:12.699508 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:12.700086 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.700599 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.701663 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.701886 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.701976 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:12.705133 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:12.705135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:12.705135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:12.705143 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:12.706915 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:12.707427 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:12.707428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:12.707429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:12.707489 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:12.707753 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:12.707754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:12.707754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:12.707888 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:12.707957 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:12.708341 (+ 384us) server_negotiation.cc:299] Negotiation successful
0924 21:19:12.708431 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:19:12.708709 1044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:12.698085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45565 (local address 127.20.115.194:43427)
0924 21:19:12.698580 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:12.698626 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:12.699306 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:12.699580 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:12.699584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:12.699788 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:12.699979 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:12.699983 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.700705 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.700706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:12.701567 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:12.701569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.701999 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.702000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:12.702155 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:12.703407 (+ 1252us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:12.703418 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:12.705032 (+ 1614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:12.707015 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:12.707022 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:12.707024 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:12.707334 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:12.707556 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:12.707556 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:12.707557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:12.707680 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:12.707972 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:12.707978 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:12.708163 (+ 185us) client_negotiation.cc:769] Sending connection context
0924 21:19:12.708290 (+ 127us) client_negotiation.cc:240] Negotiation successful
0924 21:19:12.708463 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":83,"threads_started":1}
I20260924 21:19:12.709233 1042 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45565
I20260924 21:19:12.709332 1042 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:12.709532 1042 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
I20260924 21:19:12.709940 693 ts_manager.cc:194] Registered new tserver with Master: 9558cfdfa1884161a631d51a67557350 (127.20.115.194:35989)
I20260924 21:19:12.710444 693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:43427
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:19:12.733022 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:48967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45565
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:12.856348 1048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:12.856722 1048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:12.856830 1048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:12.858589 1048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:12.858808 1048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:12.858899 1048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:19:12.861135 1048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.115.212:48967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:12.861585 1048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:12.861897 1048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:12.869200 1054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:12.869565 1048 server_base.cc:1034] running on GCE node
W20260924 21:19:12.869787 1053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:12.870044 1056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:12.870318 1048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:12.870582 1048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:12.871738 1048 hybrid_clock.cc:648] HybridClock initialized: now 1790284752871718 us; error 32 us; skew 500 ppm
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:12.875198 1048 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:19:12.875681 1048 webserver.cc:460] Webserver started at http://127.20.115.195:46617/ using document root <none> and password file <none>
I20260924 21:19:12.875918 1048 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:12.875964 1048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:12.876096 1048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:12.897701 1048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "b88b091fe5df474e81ad77bca7d0c051"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "b20e8babb0cac3ddb3b7903b333e272b"
server_key_iv: "ac262c66b17b05640bb84b4ddd12a9fc"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:12.898270 1048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "b88b091fe5df474e81ad77bca7d0c051"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "b20e8babb0cac3ddb3b7903b333e272b"
server_key_iv: "ac262c66b17b05640bb84b4ddd12a9fc"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:12.901491 1048 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260924 21:19:12.927728 1063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:12.927995 1048 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.001s
I20260924 21:19:12.928139 1048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "b88b091fe5df474e81ad77bca7d0c051"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "b20e8babb0cac3ddb3b7903b333e272b"
server_key_iv: "ac262c66b17b05640bb84b4ddd12a9fc"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:12.928308 1048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:12.939087 1048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:12.941615 1048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:12.941795 1048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:12.942081 1048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:12.942453 1048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:12.942531 1048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:12.942597 1048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:12.942655 1048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:12.954085 1048 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:41679
I20260924 21:19:12.954574 1048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:19:12.956915 1176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:41679 every 8 connection(s)
Sep 24 21:19:12 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284752, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:12.964385 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 1048
I20260924 21:19:12.964490 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:19:12.964681 20943 external_mini_cluster.cc:1428] Setting key 9824a1819ae0e9f7999dba1119140d01
I20260924 21:19:12.968164 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:12.957617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:43269 (local address 127.20.115.254:45565)
0924 21:19:12.957665 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:12.957667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:12.957680 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:19:12.957963 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:12.957964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:12.957977 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:12.958008 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:12.958821 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.959564 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.960981 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.961262 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.961302 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:12.963822 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:12.963824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:12.963825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:12.963834 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:12.966376 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:12.967006 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:12.967007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:12.967008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:12.967080 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:12.967389 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:12.967390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:12.967390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:12.967538 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:12.967616 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:12.967995 (+ 379us) server_negotiation.cc:299] Negotiation successful
0924 21:19:12.968079 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:19:12.968451 1179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:12.955395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45565 (local address 127.20.115.195:43269)
0924 21:19:12.957217 (+ 1822us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:12.957257 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:12.957838 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:12.958220 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:12.958224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:12.958459 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:12.958717 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:12.958723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.959682 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.959682 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:12.960885 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:12.960888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:12.961402 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:12.961403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:12.961537 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:12.962127 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:12.962137 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:12.963705 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:12.966468 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:12.966474 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:12.966477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:12.966923 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:12.967158 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:12.967159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:12.967159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:12.967296 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:12.967648 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:12.967654 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:12.967815 (+ 161us) client_negotiation.cc:769] Sending connection context
0924 21:19:12.967939 (+ 124us) client_negotiation.cc:240] Negotiation successful
0924 21:19:12.968228 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1781,"thread_start_us":77,"threads_started":1}
I20260924 21:19:12.969035 1177 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45565
I20260924 21:19:12.969135 1177 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:12.969348 1177 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
I20260924 21:19:12.969735 693 ts_manager.cc:194] Registered new tserver with Master: b88b091fe5df474e81ad77bca7d0c051 (127.20.115.195:41679)
I20260924 21:19:12.970213 693 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:43269
I20260924 21:19:12.978591 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:19:12.979311 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 774
I20260924 21:19:12.983969 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 911
I20260924 21:19:12.989180 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 1048
I20260924 21:19:12.995715 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 661
I20260924 21:19:13.002749 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45565
--webserver_interface=127.20.115.254
--webserver_port=33881
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:45565
--ranger_config_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:48967/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.115.212:48967/kms with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:13.082744 1180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:13.082947 1180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:13.082969 1180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:13.084129 1180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:13.084175 1180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:13.084189 1180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:13.084201 1180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:13.084213 1180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:13.085861 1180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.115.212:48967/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:45565
--ranger_config_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=33881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:13.086115 1180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:13.086364 1180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:13.091934 1185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:13.091989 1188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:13.091989 1186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:13.092304 1180 server_base.cc:1034] running on GCE node
I20260924 21:19:13.092676 1180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:13.093012 1180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:13.094180 1180 hybrid_clock.cc:648] HybridClock initialized: now 1790284753094166 us; error 28 us; skew 500 ppm
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:13.097136 1180 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:19:13.097613 1180 webserver.cc:460] Webserver started at http://127.20.115.254:33881/ using document root <none> and password file <none>
I20260924 21:19:13.097852 1180 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:13.097913 1180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:13.144973 1180 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:13.145771 1196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.145984 1180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:13.146088 1180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "68385bf56e8f44babe6219a605322b1a"
format_stamp: "Formatted at 2026-09-24 21:19:11 on dist-test-slave-cxpd"
server_key: "6241e0e626a1ca0a0d402a08f511bbf7"
server_key_iv: "f48a6b583b64dcd06ee16d43164387ad"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:13.146482 1180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:13.158222 1180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:13.160036 1180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:13.160202 1180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:13.164378 1180 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:45565
I20260924 21:19:13.164417 1248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:45565 every 8 connection(s)
I20260924 21:19:13.164837 1180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:19:13.166081 1249 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:19:13.166762 1249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Bootstrap starting.
I20260924 21:19:13.168931 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 1180
I20260924 21:19:13.169209 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:48967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:35485
--local_ip_for_outbound_sockets=127.20.115.193
--tserver_master_addrs=127.20.115.254:45565
--webserver_port=44037
--webserver_interface=127.20.115.193
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.115.212:48967/kms with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:19:13.175040 1249 log.cc:824] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:13.176957 1249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:19:13.177254 1249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Bootstrap complete.
I20260924 21:19:13.179647 1249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } }
I20260924 21:19:13.179921 1249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68385bf56e8f44babe6219a605322b1a, State: Initialized, Role: FOLLOWER
I20260924 21:19:13.180135 1249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } }
I20260924 21:19:13.180231 1249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:19:13.180284 1249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:19:13.180330 1249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:19:13.181422 1249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } }
I20260924 21:19:13.181543 1249 leader_election.cc:302] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 68385bf56e8f44babe6219a605322b1a; no voters:
I20260924 21:19:13.181761 1249 leader_election.cc:288] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 21:19:13.181795 1253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:19:13.181898 1253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [term 2 LEADER]: Becoming Leader. State: Replica: 68385bf56e8f44babe6219a605322b1a, State: Running, Role: LEADER
I20260924 21:19:13.182037 1249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:19:13.182052 1253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } }
I20260924 21:19:13.182523 1254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "68385bf56e8f44babe6219a605322b1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } } }
I20260924 21:19:13.182612 1254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:13.182788 1255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 68385bf56e8f44babe6219a605322b1a. Latest consensus state: current_term: 2 leader_uuid: "68385bf56e8f44babe6219a605322b1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "68385bf56e8f44babe6219a605322b1a" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45565 } } }
I20260924 21:19:13.182859 1255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:13.183640 1249 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260924 21:19:13.185961 1261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:19:13.187141 1261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:19:13.187428 1261 catalog_manager.cc:1259] Loaded cluster ID: bd1651d273b34afd8b3f07102fe8fa9f
I20260924 21:19:13.187546 1261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:19:13.189047 1261 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:19:13.189273 1261 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Loaded TSK: 0
I20260924 21:19:13.189538 1261 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 21:19:13.282505 1251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:13.282761 1251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:13.282794 1251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:13.284574 1251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:13.284678 1251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:13.284762 1251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:19:13.286960 1251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.115.212:48967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:35485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=44037
--tserver_master_addrs=127.20.115.254:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:13.287472 1251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:13.287771 1251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:13.293828 1286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:13.294188 1289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:13.294413 1287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:13.295039 1251 server_base.cc:1034] running on GCE node
I20260924 21:19:13.295255 1251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:13.295519 1251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:13.296700 1251 hybrid_clock.cc:648] HybridClock initialized: now 1790284753296681 us; error 33 us; skew 500 ppm
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:13.299911 1251 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:19:13.300426 1251 webserver.cc:460] Webserver started at http://127.20.115.193:44037/ using document root <none> and password file <none>
I20260924 21:19:13.300680 1251 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:13.300762 1251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:13.325170 1251 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 21:19:13.327913 1297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.328089 1251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260924 21:19:13.328176 1251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "1feb7a1a2ebc47ad8e1facab9a255b03"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "229f16a16ac95e2a85a5385392e3f1f0"
server_key_iv: "68fec6df8867f9ad0c87f48935e5cd8d"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:13.328572 1251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:13.360450 1251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:13.362871 1251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:13.363039 1251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:13.363284 1251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:13.363648 1251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:13.363699 1251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.363732 1251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:13.363756 1251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.371042 1251 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:35485
I20260924 21:19:13.371136 1410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:35485 every 8 connection(s)
I20260924 21:19:13.371505 1251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:19:13.376315 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 1251
I20260924 21:19:13.376623 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:48967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:35989
--local_ip_for_outbound_sockets=127.20.115.194
--tserver_master_addrs=127.20.115.254:45565
--webserver_port=37495
--webserver_interface=127.20.115.194
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.115.212:48967/kms with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:13.386093 1414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:13.373027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:54153 (local address 127.20.115.254:45565)
0924 21:19:13.374487 (+ 1460us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:13.374495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:13.374569 (+ 74us) server_negotiation.cc:407] Connection header received
0924 21:19:13.374720 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:13.374727 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:13.374783 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:13.374937 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:13.375736 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.377163 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.378649 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.379261 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.379367 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:13.382017 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:13.382022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:13.382022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:13.382064 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:13.384169 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:13.384696 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:13.384700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:13.384700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:13.384834 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:13.385112 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:13.385113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:13.385113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:13.385353 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:13.385470 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:13.385757 (+ 287us) server_negotiation.cc:299] Negotiation successful
0924 21:19:13.385875 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1405,"thread_start_us":83,"threads_started":1}
I20260924 21:19:13.386103 1413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:13.372948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45565 (local address 127.20.115.193:54153)
0924 21:19:13.373127 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:13.373167 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:13.373766 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:13.375096 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:13.375100 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:13.375327 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:13.375602 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:13.375608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.377286 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.377287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:13.378545 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:13.378549 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.379484 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.379486 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:13.379629 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:13.380253 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:13.380263 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:13.381881 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:13.384269 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:13.384276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:13.384278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:13.384595 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:13.384908 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:13.384909 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:13.384909 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:13.385031 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:13.385449 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:13.385455 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:13.385591 (+ 136us) client_negotiation.cc:769] Sending connection context
0924 21:19:13.385707 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 21:19:13.385874 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260924 21:19:13.386813 1411 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45565
I20260924 21:19:13.386929 1411 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:13.387105 1411 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
W20260924 21:19:13.387954 1411 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:45565 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 21:19:13.388028 1411 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:13.388094 1411 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
I20260924 21:19:13.388386 1411 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:13.388489 1411 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
W20260924 21:19:13.388764 1411 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:19:13.495846 1415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:13.496066 1415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:13.496104 1415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:13.497704 1415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:13.497814 1415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:13.497879 1415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:19:13.499418 1415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.115.212:48967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:35989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=37495
--tserver_master_addrs=127.20.115.254:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:13.499686 1415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:13.499954 1415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:13.506179 1421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:13.506516 1423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:13.507337 1415 server_base.cc:1034] running on GCE node
W20260924 21:19:13.508967 1420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:13.509258 1415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:13.509606 1415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:13.510794 1415 hybrid_clock.cc:648] HybridClock initialized: now 1790284753510781 us; error 31 us; skew 500 ppm
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:13.513975 1415 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:19:13.514513 1415 webserver.cc:460] Webserver started at http://127.20.115.194:37495/ using document root <none> and password file <none>
I20260924 21:19:13.514770 1415 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:13.514850 1415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:13.533777 1415 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260924 21:19:13.534862 1431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.535090 1415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:19:13.535168 1415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "9558cfdfa1884161a631d51a67557350"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "6849de5db38f9c7dd8bb0668049828a6"
server_key_iv: "81d29b5589df082af3315d459f9330aa"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:13.535616 1415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:13.552438 1415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:13.555080 1415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:13.555253 1415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:13.555559 1415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:13.555938 1415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:13.555992 1415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.556025 1415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:13.556078 1415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.563498 1415 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:35989
I20260924 21:19:13.564065 1415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:19:13.564949 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 1415
I20260924 21:19:13.565227 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.115.212:48967/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:41679
--local_ip_for_outbound_sockets=127.20.115.195
--tserver_master_addrs=127.20.115.254:45565
--webserver_port=46617
--webserver_interface=127.20.115.195
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.115.212:48967/kms with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:19:13.567013 1544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:35989 every 8 connection(s)
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:13.576525 1414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:13.564913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:46961 (local address 127.20.115.254:45565)
0924 21:19:13.564973 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:13.564976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:13.567603 (+ 2627us) server_negotiation.cc:407] Connection header received
0924 21:19:13.568241 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:13.568242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:13.568254 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:13.568397 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:13.569049 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.569550 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.571029 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.571233 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.571320 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:13.573487 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:13.573489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:13.573489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:13.573497 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:13.574943 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:13.575461 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:13.575462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:13.575463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:13.575524 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:13.575794 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:13.575794 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:13.575795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:13.575937 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:13.575982 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:13.576330 (+ 348us) server_negotiation.cc:299] Negotiation successful
0924 21:19:13.576432 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:19:13.577032 1547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:13.565099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45565 (local address 127.20.115.194:46961)
0924 21:19:13.567415 (+ 2316us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:13.567465 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:13.568118 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:13.568488 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:13.568492 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:13.568665 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:13.568953 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:13.568958 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.569647 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.569648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:13.570934 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:13.570937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.571351 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.571353 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:13.571487 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:13.572041 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:13.572050 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:13.573390 (+ 1340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:13.575057 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:13.575065 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:13.575067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:13.575364 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:13.575597 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:13.575598 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:13.575598 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:13.575723 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:13.576038 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:13.576044 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:13.576194 (+ 150us) client_negotiation.cc:769] Sending connection context
0924 21:19:13.576310 (+ 116us) client_negotiation.cc:240] Negotiation successful
0924 21:19:13.576462 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2272,"thread_start_us":81,"threads_started":1}
I20260924 21:19:13.577517 1545 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45565
I20260924 21:19:13.577641 1545 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:13.577833 1545 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
W20260924 21:19:13.578487 1545 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:45565 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 21:19:13.578569 1545 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:13.578644 1545 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
I20260924 21:19:13.578922 1545 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:13.579005 1545 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
W20260924 21:19:13.579273 1545 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:19:13.687565 1548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:13.687865 1548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:13.687908 1548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:13.689919 1548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:13.690021 1548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:13.690117 1548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:19:13.692449 1548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.115.212:48967/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:41679
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=46617
--tserver_master_addrs=127.20.115.254:45565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:13.693010 1548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:13.693363 1548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:13.699911 1555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:13.699936 1557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:13.700964 1554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:13.701097 1548 server_base.cc:1034] running on GCE node
I20260924 21:19:13.701454 1548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:13.701786 1548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:13.709177 1548 hybrid_clock.cc:648] HybridClock initialized: now 1790284753709135 us; error 48 us; skew 500 ppm
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:13.712515 1548 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:19:13.713095 1548 webserver.cc:460] Webserver started at http://127.20.115.195:46617/ using document root <none> and password file <none>
I20260924 21:19:13.713392 1548 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:13.713476 1548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:13.729131 1548 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 21:19:13.730846 1565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.731060 1548 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 21:19:13.731170 1548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "b88b091fe5df474e81ad77bca7d0c051"
format_stamp: "Formatted at 2026-09-24 21:19:12 on dist-test-slave-cxpd"
server_key: "b20e8babb0cac3ddb3b7903b333e272b"
server_key_iv: "ac262c66b17b05640bb84b4ddd12a9fc"
server_key_version: "kuduclusterkey@0"
I20260924 21:19:13.731592 1548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:13.771363 1548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:13.773826 1548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:13.774036 1548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:13.774338 1548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:13.774741 1548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:13.774824 1548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.774896 1548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:13.774971 1548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:13.781927 1548 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:41679
I20260924 21:19:13.782054 1678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:41679 every 8 connection(s)
I20260924 21:19:13.782352 1548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:19:13.782840 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 1548
Sep 24 21:19:13 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284753, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:13.792660 1414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:13.783486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:35245 (local address 127.20.115.254:45565)
0924 21:19:13.783620 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:13.783622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:13.783631 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:19:13.784093 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:13.784094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:13.784107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:13.784153 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:13.784692 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.785312 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.786294 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.786502 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.786576 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:13.789449 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:13.789450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:13.789451 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:13.789458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:13.791123 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:13.791626 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:13.791628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:13.791628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:13.791689 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:13.791963 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:13.791964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:13.791964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:13.792085 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:13.792126 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:13.792471 (+ 345us) server_negotiation.cc:299] Negotiation successful
0924 21:19:13.792551 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260924 21:19:13.793291 1681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:13.783182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45565 (local address 127.20.115.195:35245)
0924 21:19:13.783327 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:13.783367 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:13.783985 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:13.784220 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:13.784224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:13.784417 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:13.784599 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:13.784602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.785407 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.785408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:13.786194 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:13.786196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.786600 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.786601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:13.786719 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:13.787223 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:13.787231 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:13.789348 (+ 2117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:13.791214 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:13.791220 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:13.791222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:13.791515 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:13.791764 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:13.791764 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:13.791765 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:13.791887 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:13.792172 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:13.792177 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:13.792325 (+ 148us) client_negotiation.cc:769] Sending connection context
0924 21:19:13.792906 (+ 581us) client_negotiation.cc:240] Negotiation successful
0924 21:19:13.793071 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":81,"threads_started":1}
I20260924 21:19:13.793864 1679 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45565
I20260924 21:19:13.793965 1679 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:13.794162 1679 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
W20260924 21:19:13.794808 1679 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:45565 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260924 21:19:13.794885 1679 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:13.794967 1679 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
I20260924 21:19:13.795296 1679 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:13.795378 1679 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
W20260924 21:19:13.795703 1679 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:19:13.798746 1683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:13.783522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39372 (local address 127.20.115.254:45565)
0924 21:19:13.788533 (+ 5011us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:13.788546 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:13.788556 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:19:13.788577 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:13.788577 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:13.788590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:13.788664 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:13.789727 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.791616 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.792741 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:13.793052 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:13.793095 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:13.795857 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:13.795858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:13.795858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:13.795865 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:13.797404 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:13.797960 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:13.797961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:13.797962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:13.798023 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:13.798219 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:13.798219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:13.798220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:13.798356 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:13.798520 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:13.798558 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 21:19:13.798651 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4990,"thread_start_us":55,"threads_started":1}
Sep 24 21:19:14 dist-test-slave-cxpd krb5kdc[32685](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790284754, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:14.388981 1411 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:14.389132 1411 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
I20260924 21:19:14.389518 1213 ts_manager.cc:194] Registered new tserver with Master: 1feb7a1a2ebc47ad8e1facab9a255b03 (127.20.115.193:35485)
I20260924 21:19:14.390545 1213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:54153
I20260924 21:19:14.579479 1545 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:14.579650 1545 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
I20260924 21:19:14.579985 1213 ts_manager.cc:194] Registered new tserver with Master: 9558cfdfa1884161a631d51a67557350 (127.20.115.194:35989)
I20260924 21:19:14.580504 1213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:46961
Sep 24 21:19:14 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284754, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:14.795890 1679 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:14.796041 1679 heartbeater.cc:511] Master 127.20.115.254:45565 requested a full tablet report, sending...
I20260924 21:19:14.796290 1213 ts_manager.cc:194] Registered new tserver with Master: b88b091fe5df474e81ad77bca7d0c051 (127.20.115.195:41679)
I20260924 21:19:14.796805 1213 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:35245
I20260924 21:19:14.804656 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:19:14.813665 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.805685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.20.115.254:45565)
0924 21:19:14.805880 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:14.805882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:14.805888 (+ 6us) server_negotiation.cc:407] Connection header received
0924 21:19:14.805909 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:14.805909 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:14.805920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:14.805985 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:14.806724 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.807478 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.808555 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.808898 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.809239 (+ 341us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.810374 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:14.810375 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:14.810375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:14.810382 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:14.812023 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:14.812442 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:14.812443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:14.812443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:14.812504 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:14.812740 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:14.812741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:14.812742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:14.812898 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:14.813443 (+ 545us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:14.813485 (+ 42us) server_negotiation.cc:299] Negotiation successful
0924 21:19:14.813567 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":66,"threads_started":1}
I20260924 21:19:14.816013 1213 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39388:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:19:14.818328 1213 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:19:14.828074 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.822617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:41679 (local address 127.0.0.1:53920)
0924 21:19:14.823065 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:14.823103 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:14.823185 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:14.823488 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:14.823491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:14.823493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:14.823555 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.823557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.824880 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.824881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.826488 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.826491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.827498 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.827500 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.827856 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.827878 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 21:19:14.827945 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 21:19:14.827993 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":99,"threads_started":1}
I20260924 21:19:14.828996 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.822780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59542 (local address 127.20.115.194:35989)
0924 21:19:14.823067 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:14.823072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:14.825046 (+ 1974us) server_negotiation.cc:407] Connection header received
0924 21:19:14.825168 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:14.825171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:14.825190 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:14.825240 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:14.825467 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.826539 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.827858 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.828552 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.828597 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.828642 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:14.828890 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 21:19:14.828928 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":76,"threads_started":1}
I20260924 21:19:14.829216 1710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.822620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:35989 (local address 127.0.0.1:59542)
0924 21:19:14.824965 (+ 2345us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:14.824975 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:14.825076 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:14.825343 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:14.825344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:14.825345 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:14.825394 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.825395 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.826643 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.826644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.827758 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.827761 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.828700 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.828701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.828764 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.828768 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:19:14.829116 (+ 348us) client_negotiation.cc:240] Negotiation successful
0924 21:19:14.829157 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2328,"thread_start_us":80,"threads_started":1}
I20260924 21:19:14.830021 1466 tablet_service.cc:1467] Processing CreateTablet for tablet e017c1d528f64c8099d9dfb74ab42649 (DEFAULT_TABLE table=test-table [id=5cd8a7790298410fbcbe072847d914bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:14.830364 1466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e017c1d528f64c8099d9dfb74ab42649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:14.830698 1709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.822804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53920 (local address 127.20.115.195:41679)
0924 21:19:14.822988 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:14.822993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:14.823192 (+ 199us) server_negotiation.cc:407] Connection header received
0924 21:19:14.823263 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:14.823265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:14.823281 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:14.823366 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:14.823673 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.824735 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.826588 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.827333 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.827413 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.827464 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:14.830583 (+ 3119us) server_negotiation.cc:299] Negotiation successful
0924 21:19:14.830625 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":65,"threads_started":1}
I20260924 21:19:14.832927 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.822931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:35485 (local address 127.0.0.1:53450)
0924 21:19:14.828566 (+ 5635us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:14.828575 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:14.828610 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:14.828905 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:14.828906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:14.828907 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:14.828935 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.828936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.830267 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.830268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.831741 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.831745 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.832703 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.832704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.832760 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.832763 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:19:14.832833 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 21:19:14.832868 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5624,"thread_start_us":68,"threads_started":1}
I20260924 21:19:14.833211 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.823019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53450 (local address 127.20.115.193:35485)
0924 21:19:14.824909 (+ 1890us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:14.824913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:14.828668 (+ 3755us) server_negotiation.cc:407] Connection header received
0924 21:19:14.828706 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:14.828708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:14.828723 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:14.828816 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:14.829005 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.830171 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.831835 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.832557 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.832626 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.832670 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:14.833085 (+ 415us) server_negotiation.cc:299] Negotiation successful
0924 21:19:14.833137 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1856,"thread_start_us":48,"threads_started":1}
I20260924 21:19:14.833359 1613 tablet_service.cc:1467] Processing CreateTablet for tablet e017c1d528f64c8099d9dfb74ab42649 (DEFAULT_TABLE table=test-table [id=5cd8a7790298410fbcbe072847d914bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:14.833660 1613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e017c1d528f64c8099d9dfb74ab42649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:14.833779 1345 tablet_service.cc:1467] Processing CreateTablet for tablet e017c1d528f64c8099d9dfb74ab42649 (DEFAULT_TABLE table=test-table [id=5cd8a7790298410fbcbe072847d914bb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:14.834060 1345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e017c1d528f64c8099d9dfb74ab42649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:14.837574 1713 tablet_bootstrap.cc:492] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03: Bootstrap starting.
I20260924 21:19:14.838563 1713 tablet_bootstrap.cc:654] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:14.838629 1714 tablet_bootstrap.cc:492] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: Bootstrap starting.
I20260924 21:19:14.838927 1713 log.cc:824] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:14.839303 1714 tablet_bootstrap.cc:654] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:14.839663 1714 log.cc:824] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:14.839804 1713 tablet_bootstrap.cc:492] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03: No bootstrap required, opened a new log
I20260924 21:19:14.839869 1713 ts_tablet_manager.cc:1397] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:19:14.840409 1714 tablet_bootstrap.cc:492] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: No bootstrap required, opened a new log
I20260924 21:19:14.840457 1714 ts_tablet_manager.cc:1397] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:19:14.842396 1713 raft_consensus.cc:348] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.842547 1713 raft_consensus.cc:374] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:14.842584 1713 raft_consensus.cc:729] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1feb7a1a2ebc47ad8e1facab9a255b03, State: Initialized, Role: FOLLOWER
I20260924 21:19:14.842795 1714 raft_consensus.cc:348] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.842943 1714 raft_consensus.cc:374] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:14.842973 1714 raft_consensus.cc:729] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9558cfdfa1884161a631d51a67557350, State: Initialized, Role: FOLLOWER
I20260924 21:19:14.843081 1714 consensus_queue.cc:260] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.843304 1714 ts_tablet_manager.cc:1428] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:19:14.843370 1713 consensus_queue.cc:260] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.843578 1545 heartbeater.cc:503] Master 127.20.115.254:45565 was elected leader, sending a full tablet report...
I20260924 21:19:14.843644 1713 ts_tablet_manager.cc:1428] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260924 21:19:14.843750 1411 heartbeater.cc:503] Master 127.20.115.254:45565 was elected leader, sending a full tablet report...
I20260924 21:19:14.848745 1712 tablet_bootstrap.cc:492] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: Bootstrap starting.
I20260924 21:19:14.849814 1712 tablet_bootstrap.cc:654] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:14.850143 1712 log.cc:824] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:14.850996 1712 tablet_bootstrap.cc:492] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: No bootstrap required, opened a new log
I20260924 21:19:14.851073 1712 ts_tablet_manager.cc:1397] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:19:14.853649 1712 raft_consensus.cc:348] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.853866 1712 raft_consensus.cc:374] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:14.853951 1712 raft_consensus.cc:729] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b88b091fe5df474e81ad77bca7d0c051, State: Initialized, Role: FOLLOWER
I20260924 21:19:14.854140 1712 consensus_queue.cc:260] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.854470 1712 ts_tablet_manager.cc:1428] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260924 21:19:14.854496 1679 heartbeater.cc:503] Master 127.20.115.254:45565 was elected leader, sending a full tablet report...
W20260924 21:19:14.874512 1412 tablet.cc:2369] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 24 21:19:14 dist-test-slave-cxpd krb5kdc[32685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284754, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for HTTP/127.20.115.212@KRBTEST.COM
I20260924 21:19:14.910707 1720 raft_consensus.cc:482] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:14.910976 1720 raft_consensus.cc:504] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.912304 1720 leader_election.cc:288] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9558cfdfa1884161a631d51a67557350 (127.20.115.194:35989), 1feb7a1a2ebc47ad8e1facab9a255b03 (127.20.115.193:35485)
I20260924 21:19:14.913499 1718 raft_consensus.cc:482] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:14.913626 1718 raft_consensus.cc:504] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.913980 1718 leader_election.cc:288] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b88b091fe5df474e81ad77bca7d0c051 (127.20.115.195:41679), 9558cfdfa1884161a631d51a67557350 (127.20.115.194:35989)
I20260924 21:19:14.916493 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.911877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:36385 (local address 127.20.115.194:35989)
0924 21:19:14.911926 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:14.911929 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:14.912563 (+ 634us) server_negotiation.cc:407] Connection header received
0924 21:19:14.912667 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:14.912668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:14.912680 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:14.912727 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:14.913472 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.914201 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.915614 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.916121 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.916165 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.916173 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:14.916378 (+ 205us) server_negotiation.cc:299] Negotiation successful
0924 21:19:14.916427 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:19:14.916951 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.911638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:35989 (local address 127.20.115.195:36385)
0924 21:19:14.912495 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:14.912507 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:14.912592 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:14.913217 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:14.913218 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:14.913220 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:14.913298 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.913300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.914315 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.914316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.915506 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.915509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.916245 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.916246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.916306 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.916310 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:19:14.916755 (+ 445us) client_negotiation.cc:240] Negotiation successful
0924 21:19:14.916828 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":117,"threads_started":1}
I20260924 21:19:14.917379 1487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e017c1d528f64c8099d9dfb74ab42649" candidate_uuid: "b88b091fe5df474e81ad77bca7d0c051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9558cfdfa1884161a631d51a67557350" is_pre_election: true
I20260924 21:19:14.917550 1487 raft_consensus.cc:2454] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b88b091fe5df474e81ad77bca7d0c051 in term 0.
I20260924 21:19:14.917870 1567 leader_election.cc:302] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9558cfdfa1884161a631d51a67557350, b88b091fe5df474e81ad77bca7d0c051; no voters:
I20260924 21:19:14.918383 1720 raft_consensus.cc:2781] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:19:14.918457 1720 raft_consensus.cc:482] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:19:14.918478 1720 raft_consensus.cc:3037] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:14.919320 1720 raft_consensus.cc:504] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.919446 1720 leader_election.cc:288] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [CANDIDATE]: Term 1 election: Requested vote from peers 9558cfdfa1884161a631d51a67557350 (127.20.115.194:35989), 1feb7a1a2ebc47ad8e1facab9a255b03 (127.20.115.193:35485)
I20260924 21:19:14.919723 1487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e017c1d528f64c8099d9dfb74ab42649" candidate_uuid: "b88b091fe5df474e81ad77bca7d0c051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9558cfdfa1884161a631d51a67557350"
I20260924 21:19:14.919812 1487 raft_consensus.cc:3037] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:14.920603 1487 raft_consensus.cc:2454] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b88b091fe5df474e81ad77bca7d0c051 in term 1.
I20260924 21:19:14.920862 1567 leader_election.cc:302] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9558cfdfa1884161a631d51a67557350, b88b091fe5df474e81ad77bca7d0c051; no voters:
I20260924 21:19:14.920974 1720 raft_consensus.cc:2781] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:19:14.921089 1720 raft_consensus.cc:686] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 1 LEADER]: Becoming Leader. State: Replica: b88b091fe5df474e81ad77bca7d0c051, State: Running, Role: LEADER
I20260924 21:19:14.921166 1720 consensus_queue.cc:237] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } }
I20260924 21:19:14.921950 1212 catalog_manager.cc:5495] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 reported cstate change: term changed from 0 to 1, leader changed from <none> to b88b091fe5df474e81ad77bca7d0c051 (127.20.115.195). New cstate: current_term: 1 leader_uuid: "b88b091fe5df474e81ad77bca7d0c051" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b88b091fe5df474e81ad77bca7d0c051" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 41679 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 } health_report { overall_health: UNKNOWN } } }
I20260924 21:19:14.925344 1711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.912270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:55883 (local address 127.20.115.193:35485)
0924 21:19:14.912358 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:14.912361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:14.917328 (+ 4967us) server_negotiation.cc:407] Connection header received
0924 21:19:14.917438 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:14.917438 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:14.917448 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:14.917479 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:14.917648 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.918312 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.924246 (+ 5934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.924816 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.924868 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.924878 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:14.925231 (+ 353us) server_negotiation.cc:299] Negotiation successful
0924 21:19:14.925279 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260924 21:19:14.925806 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.912088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:35485 (local address 127.20.115.195:55883)
0924 21:19:14.917276 (+ 5188us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:14.917288 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:14.917354 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:14.917541 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:14.917542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:14.917543 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:14.917575 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.917576 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.923117 (+ 5541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.923118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.924140 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.924143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.924932 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.924933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.924987 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.924990 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:19:14.925702 (+ 712us) client_negotiation.cc:240] Negotiation successful
0924 21:19:14.925744 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5178,"thread_start_us":80,"threads_started":1}
I20260924 21:19:14.926426 1365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e017c1d528f64c8099d9dfb74ab42649" candidate_uuid: "b88b091fe5df474e81ad77bca7d0c051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" is_pre_election: true
I20260924 21:19:14.926533 1365 raft_consensus.cc:2454] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b88b091fe5df474e81ad77bca7d0c051 in term 0.
I20260924 21:19:14.926613 1364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e017c1d528f64c8099d9dfb74ab42649" candidate_uuid: "b88b091fe5df474e81ad77bca7d0c051" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03"
I20260924 21:19:14.926652 1364 raft_consensus.cc:3037] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:14.927462 1364 raft_consensus.cc:2454] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b88b091fe5df474e81ad77bca7d0c051 in term 1.
I20260924 21:19:14.931509 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.914458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:37417 (local address 127.20.115.194:35989)
0924 21:19:14.916563 (+ 2105us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:14.916565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:14.925139 (+ 8574us) server_negotiation.cc:407] Connection header received
0924 21:19:14.925632 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:14.925633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:14.925644 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:14.925673 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:14.927861 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.928544 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.930455 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.931001 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.931053 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.931061 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:14.931385 (+ 324us) server_negotiation.cc:299] Negotiation successful
0924 21:19:14.931440 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2092,"thread_start_us":93,"threads_started":1}
I20260924 21:19:14.931919 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.914200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:35989 (local address 127.20.115.193:37417)
0924 21:19:14.925086 (+ 10886us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:14.925097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:14.925165 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:14.926018 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:14.926019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:14.926022 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:14.926093 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.926095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.928646 (+ 2551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.928647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.929778 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.929781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.931170 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.931171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.931246 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.931250 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:19:14.931747 (+ 497us) client_negotiation.cc:240] Negotiation successful
0924 21:19:14.931794 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10865,"thread_start_us":86,"threads_started":1}
I20260924 21:19:14.932471 1487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e017c1d528f64c8099d9dfb74ab42649" candidate_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9558cfdfa1884161a631d51a67557350" is_pre_election: true
I20260924 21:19:14.932582 1487 raft_consensus.cc:2379] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1feb7a1a2ebc47ad8e1facab9a255b03 in current term 1: Already voted for candidate b88b091fe5df474e81ad77bca7d0c051 in this term.
I20260924 21:19:14.934260 1709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.914840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:53823 (local address 127.20.115.195:41679)
0924 21:19:14.914877 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:14.914879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:14.927416 (+ 12537us) server_negotiation.cc:407] Connection header received
0924 21:19:14.927546 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:14.927547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:14.927559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:14.927592 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:14.930181 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.930917 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.933238 (+ 2321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.933839 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.933895 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.933904 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:14.934152 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 21:19:14.934196 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:19:14.934719 1726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.914686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:41679 (local address 127.20.115.193:53823)
0924 21:19:14.927337 (+ 12651us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:14.927351 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:14.927447 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:14.927660 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:14.927660 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:14.927662 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:14.927735 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.927738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.931037 (+ 3299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.931038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.932321 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:14.932324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.933974 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.933975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:14.934054 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.934058 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:19:14.934583 (+ 525us) client_negotiation.cc:240] Negotiation successful
0924 21:19:14.934633 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12629,"thread_start_us":72,"threads_started":1}
I20260924 21:19:14.935204 1627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e017c1d528f64c8099d9dfb74ab42649" candidate_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b88b091fe5df474e81ad77bca7d0c051" is_pre_election: true
I20260924 21:19:14.935622 1301 leader_election.cc:302] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1feb7a1a2ebc47ad8e1facab9a255b03; no voters: 9558cfdfa1884161a631d51a67557350, b88b091fe5df474e81ad77bca7d0c051
I20260924 21:19:14.935853 1718 raft_consensus.cc:2726] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260924 21:19:14.944118 1709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:14.939702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53936 (local address 127.20.115.195:41679)
0924 21:19:14.939733 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:14.939735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:14.939741 (+ 6us) server_negotiation.cc:407] Connection header received
0924 21:19:14.939772 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:14.939772 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:14.939789 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:14.939844 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:19:14.940065 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.940767 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.943112 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:14.943383 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:14.943455 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:14.943645 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:19:14.943767 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:19:14.943832 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:14.944038 (+ 206us) server_negotiation.cc:299] Negotiation successful
0924 21:19:14.944056 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:19:14.946190 1364 raft_consensus.cc:1261] T e017c1d528f64c8099d9dfb74ab42649 P 1feb7a1a2ebc47ad8e1facab9a255b03 [term 1 FOLLOWER]: Refusing update from remote peer b88b091fe5df474e81ad77bca7d0c051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:14.946520 1727 consensus_queue.cc:1035] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1feb7a1a2ebc47ad8e1facab9a255b03" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 35485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:14.948491 1487 raft_consensus.cc:1261] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 1 FOLLOWER]: Refusing update from remote peer b88b091fe5df474e81ad77bca7d0c051: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:14.948820 1727 consensus_queue.cc:1035] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9558cfdfa1884161a631d51a67557350" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 35989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:14.952485 1731 mvcc.cc:205] Tried to move back new op lower bound from 7333006356256952320 to 7333006356157558784. Current Snapshot: MvccSnapshot[applied={T|T < 7333006356256952320}]
I20260924 21:19:14.953095 1732 mvcc.cc:205] Tried to move back new op lower bound from 7333006356256952320 to 7333006356157558784. Current Snapshot: MvccSnapshot[applied={T|T < 7333006356256952320}]
I20260924 21:19:14.954531 1734 mvcc.cc:205] Tried to move back new op lower bound from 7333006356256952320 to 7333006356157558784. Current Snapshot: MvccSnapshot[applied={T|T < 7333006356256952320}]
I20260924 21:19:14.956990 1212 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:19:14.957079 1212 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39388:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:19:14.957855 1212 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 68385bf56e8f44babe6219a605322b1a: Sending DeleteTablet for 3 replicas of tablet e017c1d528f64c8099d9dfb74ab42649
I20260924 21:19:14.958356 1613 tablet_service.cc:1514] Processing DeleteTablet for tablet e017c1d528f64c8099d9dfb74ab42649 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:14 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:53920
I20260924 21:19:14.958514 1345 tablet_service.cc:1514] Processing DeleteTablet for tablet e017c1d528f64c8099d9dfb74ab42649 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:14 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:53450
I20260924 21:19:14.958590 1466 tablet_service.cc:1514] Processing DeleteTablet for tablet e017c1d528f64c8099d9dfb74ab42649 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:14 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:59542
I20260924 21:19:14.959208 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 1251
I20260924 21:19:14.964862 1740 tablet_replica.cc:330] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: stopping tablet replica
I20260924 21:19:14.965034 1740 raft_consensus.cc:2229] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:19:14.965232 1740 raft_consensus.cc:2258] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:14.965765 1740 ts_tablet_manager.cc:1905] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:19:14.965945 1742 tablet_replica.cc:330] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: stopping tablet replica
I20260924 21:19:14.966087 1742 raft_consensus.cc:2229] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:19:14.966255 1742 raft_consensus.cc:2258] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:14.968410 1742 ts_tablet_manager.cc:1905] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:19:14.970115 1742 ts_tablet_manager.cc:1918] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:19:14.974150 1740 ts_tablet_manager.cc:1918] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:19:14.974284 1740 log.cc:1196] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/e017c1d528f64c8099d9dfb74ab42649
I20260924 21:19:14.974630 1740 ts_tablet_manager.cc:1939] T e017c1d528f64c8099d9dfb74ab42649 P b88b091fe5df474e81ad77bca7d0c051: Deleting consensus metadata
I20260924 21:19:14.981274 1200 catalog_manager.cc:4841] TS b88b091fe5df474e81ad77bca7d0c051 (127.20.115.195:41679): tablet e017c1d528f64c8099d9dfb74ab42649 (table test-table [id=5cd8a7790298410fbcbe072847d914bb]) successfully deleted
I20260924 21:19:14.981503 1742 log.cc:1196] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/e017c1d528f64c8099d9dfb74ab42649
I20260924 21:19:14.981869 1742 ts_tablet_manager.cc:1939] T e017c1d528f64c8099d9dfb74ab42649 P 9558cfdfa1884161a631d51a67557350: Deleting consensus metadata
I20260924 21:19:14.982378 1198 catalog_manager.cc:4841] TS 9558cfdfa1884161a631d51a67557350 (127.20.115.194:35989): tablet e017c1d528f64c8099d9dfb74ab42649 (table test-table [id=5cd8a7790298410fbcbe072847d914bb]) successfully deleted
W20260924 21:19:14.982959 1199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:35485 (error 108)
I20260924 21:19:14.992965 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 1415
I20260924 21:19:15.004814 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 1548
I20260924 21:19:15.010747 1710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:15.010006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:35485 (local address 127.0.0.1:53460)
0924 21:19:15.010638 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:15.010700 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:35485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":602}
W20260924 21:19:15.011035 1199 catalog_manager.cc:4584] TS 1feb7a1a2ebc47ad8e1facab9a255b03 (127.20.115.193:35485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e017c1d528f64c8099d9dfb74ab42649: Network error: Client connection negotiation failed: client connection to 127.20.115.193:35485: connect: Connection refused (error 111)
I20260924 21:19:15.013396 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 1180
I20260924 21:19:15.020967 20943 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260924 21:19:15.365320 20943 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260924 21:19:15.365419 20943 mini_ranger.cc:66] Stopping Ranger...
I20260924 21:19:15.709909 20943 mini_ranger.cc:68] Stopped Ranger
2026-09-24 21:19:15.710 UTC [32715] LOG: received smart shutdown request
2026-09-24 21:19:15.713 UTC [32715] LOG: background worker "logical replication launcher" (PID 32724) exited with exit code 1
2026-09-24 21:19:15.713 UTC [32719] LOG: shutting down
2026-09-24 21:19:15.894 UTC [32715] LOG: database system is shut down
2026-09-24T21:19:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47215 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:19:16 dist-test-slave-cxpd krb5kdc[1752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:19:16 dist-test-slave-cxpd krb5kdc[1752](info): set up 2 sockets
Sep 24 21:19:16 dist-test-slave-cxpd krb5kdc[1752](info): commencing operation
krb5kdc: starting...
W20260924 21:19:17.048660 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:19:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:19:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:19:17.167866 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46497
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:46497,127.20.115.253:45293,127.20.115.252:33257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:17.248347 1765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:17.248613 1765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:17.248651 1765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:17.249861 1765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:17.249949 1765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:17.249974 1765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:17.249996 1765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:17.250025 1765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:17.251518 1765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:46497,127.20.115.253:45293,127.20.115.252:33257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:17.251797 1765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:17.252029 1765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:17.256877 1770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:17.256873 1773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:17.256918 1771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:17.257313 1765 server_base.cc:1034] running on GCE node
I20260924 21:19:17.257511 1765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:17.257737 1765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:17.258898 1765 hybrid_clock.cc:648] HybridClock initialized: now 1790284757258875 us; error 36 us; skew 500 ppm
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:17.261377 1765 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:19:17.261826 1765 webserver.cc:460] Webserver started at http://127.20.115.254:41295/ using document root <none> and password file <none>
I20260924 21:19:17.262040 1765 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:17.262115 1765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:17.262264 1765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:17.263329 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "9f791a4748c7477e98829b84846a8993"
format_stamp: "Formatted at 2026-09-24 21:19:17 on dist-test-slave-cxpd"
server_key: "da8c0ee91f17ee38a71f9a698e91b5eb"
server_key_iv: "7abc882ddd965c69397b8fa92ad574bc"
server_key_version: "encryptionkey@0"
I20260924 21:19:17.263688 1765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "9f791a4748c7477e98829b84846a8993"
format_stamp: "Formatted at 2026-09-24 21:19:17 on dist-test-slave-cxpd"
server_key: "da8c0ee91f17ee38a71f9a698e91b5eb"
server_key_iv: "7abc882ddd965c69397b8fa92ad574bc"
server_key_version: "encryptionkey@0"
I20260924 21:19:17.265069 1765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:19:17.265810 1780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:17.266060 1765 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:17.266178 1765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "9f791a4748c7477e98829b84846a8993"
format_stamp: "Formatted at 2026-09-24 21:19:17 on dist-test-slave-cxpd"
server_key: "da8c0ee91f17ee38a71f9a698e91b5eb"
server_key_iv: "7abc882ddd965c69397b8fa92ad574bc"
server_key_version: "encryptionkey@0"
I20260924 21:19:17.266260 1765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:17.281894 1765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:17.283747 1765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:17.283921 1765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:17.288105 1765 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:46497
I20260924 21:19:17.288142 1832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:46497 every 8 connection(s)
I20260924 21:19:17.288479 1765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:19:17.289347 1833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:17.290860 1833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:19:17.292644 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 1765
I20260924 21:19:17.292764 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:19:17.292974 20943 external_mini_cluster.cc:1428] Setting key f0a624c3353dc4128d35b043a4bb9fc1
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.115.253@KRBTEST.COM; defaulting to no policy
I20260924 21:19:17.301399 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.291667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.20.115.254:46497)
0924 21:19:17.291805 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.291809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.291845 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:19:17.292311 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.292313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.292330 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.292407 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.293053 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.294329 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.295287 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.295613 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.295728 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.298185 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.298198 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.298198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.298204 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.299740 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.300134 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.300137 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.300138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.300253 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.300484 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.300485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.300485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.300697 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.300788 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.301003 (+ 215us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.301158 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":98,"thread_start_us":70,"threads_started":1}
I20260924 21:19:17.301399 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.291508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46497 (local address 127.0.0.1:49864)
0924 21:19:17.291663 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.291698 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:17.292177 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:17.292411 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:17.292413 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:17.292660 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:17.292941 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.292947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.294439 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.294440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.295196 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.295199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.295700 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.295701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.295764 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.296311 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:17.296319 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:17.298090 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:17.299843 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.299849 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.299851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.300051 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.300329 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.300329 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.300330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.300419 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.300768 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:17.300773 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:17.300849 (+ 76us) client_negotiation.cc:769] Sending connection context
0924 21:19:17.300956 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 21:19:17.301203 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":117,"thread_start_us":65,"threads_started":1}
I20260924 21:19:17.302281 1833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:19:17.302616 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.302511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:45293 (local address 127.0.0.1:53546)
0924 21:19:17.302556 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.302596 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.253@KRBTEST.COM" created.
W20260924 21:19:17.302774 1781 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
I20260924 21:19:17.303277 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.303157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:45293 (local address 127.0.0.1:53552)
0924 21:19:17.303206 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.303242 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 21:19:17.303480 1833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.253:45293: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab.
Entry for principal kudu/127.20.115.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab.
I20260924 21:19:17.317711 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.253:45293
--webserver_interface=127.20.115.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab
--principal=kudu/127.20.115.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:46497,127.20.115.253:45293,127.20.115.252:33257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:19:17.352742 1833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } attempt: 1
I20260924 21:19:17.353325 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.353189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:45293 (local address 127.0.0.1:53566)
0924 21:19:17.353254 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.353294 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260924 21:19:17.353677 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.353573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:45293 (local address 127.0.0.1:53574)
0924 21:19:17.353620 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.353646 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 21:19:17.353790 1833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.253:45293: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
W20260924 21:19:17.399533 1840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:17.399812 1840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:17.399855 1840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:17.401062 1840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:17.401145 1840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:17.401170 1840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:17.401191 1840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:17.401223 1840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:17.402751 1840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:46497,127.20.115.253:45293,127.20.115.252:33257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab
--principal=kudu/127.20.115.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.253:45293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.115.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:17.403059 1840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:17.403298 1840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:17.408315 1848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:17.410212 1840 server_base.cc:1034] running on GCE node
W20260924 21:19:17.410357 1846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:17.410370 1845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:17.410725 1840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:17.411070 1840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:17.412267 1840 hybrid_clock.cc:648] HybridClock initialized: now 1790284757412256 us; error 39 us; skew 500 ppm
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:17.415009 1840 init.cc:377] Logged in from keytab as kudu/127.20.115.253@KRBTEST.COM (short username kudu)
I20260924 21:19:17.415521 1840 webserver.cc:460] Webserver started at http://127.20.115.253:43929/ using document root <none> and password file <none>
I20260924 21:19:17.415750 1840 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:17.415818 1840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:17.415973 1840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:17.417135 1840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/data/instance:
uuid: "11286e1302a64513a14c8467f181f51b"
format_stamp: "Formatted at 2026-09-24 21:19:17 on dist-test-slave-cxpd"
server_key: "5a24d0f665d8e74f7be9653698f4e16a"
server_key_iv: "cc0ec8dfa3a27486311689f2b5deb34b"
server_key_version: "encryptionkey@0"
I20260924 21:19:17.417505 1840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/wal/instance:
uuid: "11286e1302a64513a14c8467f181f51b"
format_stamp: "Formatted at 2026-09-24 21:19:17 on dist-test-slave-cxpd"
server_key: "5a24d0f665d8e74f7be9653698f4e16a"
server_key_iv: "cc0ec8dfa3a27486311689f2b5deb34b"
server_key_version: "encryptionkey@0"
I20260924 21:19:17.418884 1840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:19:17.419747 1855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:17.420006 1840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:17.420117 1840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/wal
uuid: "11286e1302a64513a14c8467f181f51b"
format_stamp: "Formatted at 2026-09-24 21:19:17 on dist-test-slave-cxpd"
server_key: "5a24d0f665d8e74f7be9653698f4e16a"
server_key_iv: "cc0ec8dfa3a27486311689f2b5deb34b"
server_key_version: "encryptionkey@0"
I20260924 21:19:17.420204 1840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:17.421963 1833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } attempt: 2
I20260924 21:19:17.422405 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.422262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:45293 (local address 127.0.0.1:53584)
0924 21:19:17.422327 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.422370 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 21:19:17.422806 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.422707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:45293 (local address 127.0.0.1:53598)
0924 21:19:17.422754 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.422782 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 21:19:17.423019 1833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.253:45293: Network error: Client connection negotiation failed: client connection to 127.20.115.253:45293: connect: Connection refused (error 111)
I20260924 21:19:17.433944 1840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:17.436033 1840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:17.436193 1840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:17.440433 1840 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.253:45293
I20260924 21:19:17.440465 1907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.253:45293 every 8 connection(s)
I20260924 21:19:17.440899 1840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
I20260924 21:19:17.441782 1908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:17.442621 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 1840
I20260924 21:19:17.442731 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-1/wal/instance
I20260924 21:19:17.442936 20943 external_mini_cluster.cc:1428] Setting key 700efadc4ff2cd6551c34f1cb2decb40
I20260924 21:19:17.443971 1908 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.115.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.252@KRBTEST.COM" created.
I20260924 21:19:17.454475 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.444940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49876 (local address 127.20.115.254:46497)
0924 21:19:17.444985 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.444987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.445343 (+ 356us) server_negotiation.cc:407] Connection header received
0924 21:19:17.445873 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.445875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.445892 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.445955 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.446602 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.447139 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.448164 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.448381 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.448472 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.451069 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.451071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.451071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.451080 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.452677 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.453198 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.453199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.453199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.453265 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.453628 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.453630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.453631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.453811 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.453894 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.454224 (+ 330us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.454327 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:19:17.454495 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.444931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46497 (local address 127.0.0.1:49876)
0924 21:19:17.445196 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.445235 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:17.445741 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:17.446038 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:17.446041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:17.446303 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:17.446499 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.446503 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.447245 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.447246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.448057 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.448059 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.448493 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.448494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.448627 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.449279 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:17.449291 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:17.450952 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:17.452774 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.452838 (+ 64us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.452841 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.453114 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.453353 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.453355 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.453355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.453512 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.453923 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:17.453929 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:17.454071 (+ 142us) client_negotiation.cc:769] Sending connection context
0924 21:19:17.454175 (+ 104us) client_negotiation.cc:240] Negotiation successful
0924 21:19:17.454318 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
I20260924 21:19:17.455345 1908 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
I20260924 21:19:17.464056 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.455645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:45293 (local address 127.0.0.1:53606)
0924 21:19:17.455697 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.455709 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:17.455749 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:17.456044 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:17.456045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:17.456164 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:17.456245 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.456247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.457248 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.457250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.458133 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.458136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.458867 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.458868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.458926 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.459416 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:17.459420 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:17.460614 (+ 1194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:17.462731 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.462731 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.462732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.462985 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.463246 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.463247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.463247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.463305 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.463649 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:17.463650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:17.463810 (+ 160us) client_negotiation.cc:769] Sending connection context
0924 21:19:17.463886 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:19:17.463983 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:19:17.464222 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.455704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53606 (local address 127.20.115.253:45293)
0924 21:19:17.455879 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.455884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.455908 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:19:17.455931 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.455932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.455949 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.456033 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.456383 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.457154 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.458235 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.458768 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.458857 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.460704 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.460714 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.460715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.460722 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.462634 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.463102 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.463105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.463105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.463171 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.463366 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.463367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.463367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.463532 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.463597 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.463978 (+ 381us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.464064 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260924 21:19:17.464926 1908 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:19:17.465409 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.465269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38236)
0924 21:19:17.465338 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.465386 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260924 21:19:17.465595 1859 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
I20260924 21:19:17.466178 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.466081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38250)
0924 21:19:17.466130 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.466158 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260924 21:19:17.466305 1908 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:33257: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab.
Entry for principal kudu/127.20.115.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab.
I20260924 21:19:17.471616 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.252:33257
--webserver_interface=127.20.115.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab
--principal=kudu/127.20.115.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:46497,127.20.115.253:45293,127.20.115.252:33257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:19:17.514212 1833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } attempt: 3
I20260924 21:19:17.515625 1908 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } attempt: 1
I20260924 21:19:17.516011 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.515864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38254)
0924 21:19:17.515928 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.515971 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 21:19:17.517066 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.516954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38270)
0924 21:19:17.517001 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.517039 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 21:19:17.517405 1908 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:33257: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
I20260924 21:19:17.521624 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.514693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53616 (local address 127.20.115.253:45293)
0924 21:19:17.514764 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.514766 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.514778 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:19:17.514813 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.514814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.514824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.514887 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.515260 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.515955 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.516709 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.517059 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.517136 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.519181 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.519183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.519183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.519189 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.520490 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.520905 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.520906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.520906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.520955 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.521157 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.521157 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.521157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.521260 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.521313 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.521489 (+ 176us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.521562 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:19:17.521649 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.514614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:45293 (local address 127.0.0.1:53616)
0924 21:19:17.514676 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.514685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:17.514738 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:17.514920 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:17.514921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:17.515095 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:17.515164 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.515166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.516073 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.516074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.516621 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.516622 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.517135 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.517136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.517221 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.517674 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:17.517679 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:17.519085 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:17.520584 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.520585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.520586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.520834 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.521019 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.521020 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.521020 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.521095 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.521321 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:17.521322 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:17.521385 (+ 63us) client_negotiation.cc:769] Sending connection context
0924 21:19:17.521456 (+ 71us) client_negotiation.cc:240] Negotiation successful
0924 21:19:17.521552 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260924 21:19:17.522415 1833 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:19:17.522723 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.522636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38286)
0924 21:19:17.522670 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.522700 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 21:19:17.523082 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.522983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38302)
0924 21:19:17.523036 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.523067 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260924 21:19:17.523178 1833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:33257: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
I20260924 21:19:17.554375 1833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } attempt: 1
I20260924 21:19:17.554869 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.554725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38312)
0924 21:19:17.554789 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.554834 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 21:19:17.555284 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.555185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38324)
0924 21:19:17.555229 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.555261 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 21:19:17.555415 1833 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:33257: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
W20260924 21:19:17.555330 1915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:17.555632 1915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:17.555678 1915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:17.556910 1915 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:17.556957 1915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:17.556972 1915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:17.556984 1915 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:17.556996 1915 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:17.558506 1915 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:46497,127.20.115.253:45293,127.20.115.252:33257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab
--principal=kudu/127.20.115.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.252:33257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.115.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1915
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:17.558765 1915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:17.559000 1915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:17.563874 1923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:17.563891 1921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:17.563894 1920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:17.564209 1915 server_base.cc:1034] running on GCE node
I20260924 21:19:17.564617 1915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:17.564946 1915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:17.566135 1915 hybrid_clock.cc:648] HybridClock initialized: now 1790284757566133 us; error 40 us; skew 500 ppm
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:17.568671 1915 init.cc:377] Logged in from keytab as kudu/127.20.115.252@KRBTEST.COM (short username kudu)
I20260924 21:19:17.569168 1915 webserver.cc:460] Webserver started at http://127.20.115.252:34581/ using document root <none> and password file <none>
I20260924 21:19:17.569346 1915 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:17.569386 1915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:17.569478 1915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:17.570571 1915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/data/instance:
uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e"
format_stamp: "Formatted at 2026-09-24 21:19:17 on dist-test-slave-cxpd"
server_key: "db9114fab227270128e91f945d787ed2"
server_key_iv: "7d74e0e9ab2e219e2beee193ab6511b2"
server_key_version: "encryptionkey@0"
I20260924 21:19:17.570863 1915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/wal/instance:
uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e"
format_stamp: "Formatted at 2026-09-24 21:19:17 on dist-test-slave-cxpd"
server_key: "db9114fab227270128e91f945d787ed2"
server_key_iv: "7d74e0e9ab2e219e2beee193ab6511b2"
server_key_version: "encryptionkey@0"
I20260924 21:19:17.572101 1915 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:19:17.572875 1930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:17.573076 1915 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:19:17.573145 1915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/wal
uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e"
format_stamp: "Formatted at 2026-09-24 21:19:17 on dist-test-slave-cxpd"
server_key: "db9114fab227270128e91f945d787ed2"
server_key_iv: "7d74e0e9ab2e219e2beee193ab6511b2"
server_key_version: "encryptionkey@0"
I20260924 21:19:17.573253 1915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:17.585564 1908 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } attempt: 2
I20260924 21:19:17.586037 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.585887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38326)
0924 21:19:17.585948 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.585998 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 21:19:17.586411 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.586331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38334)
0924 21:19:17.586371 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.586396 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 21:19:17.586504 1908 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:33257: Network error: Client connection negotiation failed: client connection to 127.20.115.252:33257: connect: Connection refused (error 111)
I20260924 21:19:17.589501 1915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:17.591799 1915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:17.591913 1915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:17.596068 1915 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.252:33257
I20260924 21:19:17.596101 1982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.252:33257 every 8 connection(s)
I20260924 21:19:17.596469 1915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
I20260924 21:19:17.596722 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 1915
I20260924 21:19:17.596882 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790284606872664-20943-0/minicluster-data/master-2/wal/instance
I20260924 21:19:17.597169 20943 external_mini_cluster.cc:1428] Setting key f1bb3ed0980d0d2b02c335be775254f8
I20260924 21:19:17.597565 1983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:17.599898 1983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:17.607867 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.597665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38338 (local address 127.20.115.252:33257)
0924 21:19:17.597847 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.597855 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.597880 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:19:17.598332 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.598339 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.598386 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.598497 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.599259 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.600150 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.601172 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.601696 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.601772 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.604512 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.604526 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.604526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.604533 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.606233 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.606594 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.606597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.606598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.606698 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.606870 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.606871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.606871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.607202 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.607299 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.607471 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.607614 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260924 21:19:17.608443 1987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.600474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46497 (local address 127.0.0.1:49890)
0924 21:19:17.600663 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.600701 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:17.600768 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:17.600973 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:17.600975 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:17.601312 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:17.601378 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.601380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.602120 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.602121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.602768 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.602770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.603298 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.603299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.603368 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.603863 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:17.603873 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:17.605396 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:17.607162 (+ 1766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.607176 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.607177 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.607465 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.607698 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.607699 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.607699 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.607757 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.608071 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:17.608075 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:17.608123 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 21:19:17.608234 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 21:19:17.608336 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":117,"threads_started":1}
I20260924 21:19:17.608446 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.600569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49890 (local address 127.20.115.254:46497)
0924 21:19:17.600635 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.600637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.600772 (+ 135us) server_negotiation.cc:407] Connection header received
0924 21:19:17.600874 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.600875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.600885 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.600947 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.601457 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.602015 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.602857 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.603185 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.603267 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.605484 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.605485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.605486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.605494 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.607059 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.607557 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.607558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.607559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.607617 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.607826 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.607827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.607828 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.607975 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.608044 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.608279 (+ 235us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.608345 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:19:17.608971 1983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:17.616185 1987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.609272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:45293 (local address 127.0.0.1:53628)
0924 21:19:17.609322 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.609332 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:17.609366 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:17.609620 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:17.609621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:17.609767 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:17.609796 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.609798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.610460 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.610461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.610944 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.610945 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.611421 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.611422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.611485 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.611863 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:17.611868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:17.613107 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:17.615058 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.615059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.615059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.615235 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.615449 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.615449 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.615450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.615499 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.615823 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:17.615824 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:17.615873 (+ 49us) client_negotiation.cc:769] Sending connection context
0924 21:19:17.615949 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:19:17.616032 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 21:19:17.616171 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.609353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53628 (local address 127.20.115.253:45293)
0924 21:19:17.609406 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.609408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.609417 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:19:17.609441 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.609442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.609450 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.609620 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.609886 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.610376 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.611108 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.611323 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.611421 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.613244 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.613246 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.613246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.613253 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.614960 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.615314 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.615315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.615316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.615378 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.615571 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.615572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.615572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.615721 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.615826 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.615990 (+ 164us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.616078 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 21:19:17.616613 1983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:19:17.616909 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.608801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49902 (local address 127.20.115.254:46497)
0924 21:19:17.608832 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.608834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.608862 (+ 28us) server_negotiation.cc:407] Connection header received
0924 21:19:17.608938 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.608939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.608948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.608976 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.609624 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.610081 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.610766 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.610959 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.611032 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.613896 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.613898 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.613898 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.613906 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.615565 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.615952 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.615953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.615953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.616010 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.616227 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.616228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.616228 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.616369 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.616560 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.616599 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.616672 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
I20260924 21:19:17.624235 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.616970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38354 (local address 127.20.115.252:33257)
0924 21:19:17.617066 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.617067 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.617073 (+ 6us) server_negotiation.cc:407] Connection header received
0924 21:19:17.617088 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.617089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.617098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.617136 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.617391 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.617956 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.618881 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.619092 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.619183 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.621296 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.621297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.621297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.621304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.623000 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.623432 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.623433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.623433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.623487 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.623674 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.623675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.623676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.623787 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.623860 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.623985 (+ 125us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.624151 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":83}
I20260924 21:19:17.624235 1987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.616905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38354)
0924 21:19:17.616960 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.616970 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:17.617007 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:17.617158 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:17.617159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:17.617285 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:17.617312 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.617317 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.618044 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.618044 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.618806 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.618809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.619181 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.619182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.619229 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.619654 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:17.619659 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:17.621186 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:17.623118 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.623119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.623119 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.623342 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.623566 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.623566 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.623567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.623626 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.623851 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:17.623852 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:17.623888 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 21:19:17.623955 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 21:19:17.624091 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260924 21:19:17.625839 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.617967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53632 (local address 127.20.115.253:45293)
0924 21:19:17.618003 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.618005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.618014 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:19:17.618085 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.618085 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.618092 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.618146 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.618701 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.619132 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.619944 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.620147 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.620242 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.622987 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.622988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.622988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.622996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.624643 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.625092 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.625093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.625093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.625153 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.625320 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.625321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.625321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.625456 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.625632 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.625669 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.625740 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:19:17.626426 1983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e: Bootstrap starting.
I20260924 21:19:17.627324 1983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:17.627561 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:19:17.627657 1983 log.cc:824] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:17.628470 1983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e: No bootstrap required, opened a new log
I20260924 21:19:17.630601 1833 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } attempt: 2
I20260924 21:19:17.630745 1983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } }
I20260924 21:19:17.630919 1983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:17.630980 1983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d0048bbb9134a38b0ec9a2dfdf5f27e, State: Initialized, Role: FOLLOWER
I20260924 21:19:17.631122 1983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } }
I20260924 21:19:17.631457 1983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:19:17.632308 1997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:17.632418 1997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:19:17.633625 2008 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:19:17.633675 2008 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
I20260924 21:19:17.633774 1915 master_runner.cc:186] Error getting master registration for 127.20.115.254:46497: OK, instance_id { permanent_uuid: "9f791a4748c7477e98829b84846a8993" instance_seqno: 1790284757286897 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:19:17.636250 1915 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:19:17.637480 1836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.629202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.20.115.254:46497)
0924 21:19:17.629296 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.629298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.629421 (+ 123us) server_negotiation.cc:407] Connection header received
0924 21:19:17.629490 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.629491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.629500 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.629556 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.630210 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.630661 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.631841 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.632133 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.632201 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.633575 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.633576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.633577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.633584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.636232 (+ 2648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.636595 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.636596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.636597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.636661 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.636935 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.636936 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.636937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.637076 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.637191 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.637304 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.637378 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260924 21:19:17.641103 1835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.630837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38384)
0924 21:19:17.630883 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.630893 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:17.630930 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:17.631142 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:17.631143 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:17.631304 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:17.631340 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.631342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.632297 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.632298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.633110 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.633113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.633792 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.633793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.633850 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.634348 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:17.634353 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:17.636623 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:17.638897 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.638898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.638899 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.639118 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.639361 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.639362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.639363 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.639439 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.640512 (+ 1073us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:17.640513 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:17.640563 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 21:19:17.640898 (+ 335us) client_negotiation.cc:240] Negotiation successful
0924 21:19:17.640991 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:19:17.641501 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.630874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38384 (local address 127.20.115.252:33257)
0924 21:19:17.631034 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.631036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.631043 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:19:17.631061 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.631062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.631071 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.631134 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.631419 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.632191 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.633215 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.633573 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.633632 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.636706 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.636707 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.636707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.636714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.638791 (+ 2077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.639208 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.639209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.639209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.639281 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.639513 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.639514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.639515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.639943 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.640013 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.641261 (+ 1248us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.641354 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260924 21:19:17.642871 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.629220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53634 (local address 127.20.115.253:45293)
0924 21:19:17.629257 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.629259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.634262 (+ 5003us) server_negotiation.cc:407] Connection header received
0924 21:19:17.634551 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.634552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.634563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.634622 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.635289 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.635974 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.637342 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.637628 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.637672 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.639364 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.639365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.639366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.639372 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.641130 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.642034 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.642035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.642036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.642095 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.642298 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.642299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.642299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.642415 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.642483 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.642706 (+ 223us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.642783 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:19:17.642925 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.629082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.20.115.252:33257)
0924 21:19:17.629121 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.629123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.634641 (+ 5518us) server_negotiation.cc:407] Connection header received
0924 21:19:17.634729 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.634730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.634739 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.634767 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.635408 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.636067 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.637235 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.637512 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.637561 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.639818 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.639821 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.639822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.639828 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.641502 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.642127 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.642128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.642129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.642184 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.642384 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.642384 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.642385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.642492 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.642556 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.642756 (+ 200us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.642827 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:19:17.643213 1833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993: Bootstrap starting.
I20260924 21:19:17.643965 1833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:17.644253 1833 log.cc:824] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:17.645118 1833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993: No bootstrap required, opened a new log
I20260924 21:19:17.646486 1833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } }
I20260924 21:19:17.646595 1833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:17.646617 1833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f791a4748c7477e98829b84846a8993, State: Initialized, Role: FOLLOWER
I20260924 21:19:17.646749 1833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } }
I20260924 21:19:17.647030 1833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:19:17.647017 2011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:17.647140 2011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:19:17.648885 2022 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:19:17.649501 2022 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:19:17.649665 1765 master_runner.cc:186] Error getting master registration for 127.20.115.253:45293: OK, instance_id { permanent_uuid: "11286e1302a64513a14c8467f181f51b" instance_seqno: 1790284757439261 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:19:17.652002 1765 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:19:17.677670 1908 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } attempt: 3
Sep 24 21:19:17 dist-test-slave-cxpd krb5kdc[1752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284757, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
I20260924 21:19:17.685889 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.677954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:33257 (local address 127.0.0.1:38392)
0924 21:19:17.678016 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:17.678028 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:17.678078 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:17.678209 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:17.678209 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:17.678403 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:17.678456 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.678458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.679382 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.679384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.680191 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:17.680193 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.680671 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.680672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:17.680732 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.681267 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:17.681272 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:17.682994 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:17.684762 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.684764 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.684764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.685006 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.685216 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:17.685217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:17.685218 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:17.685286 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:17.685567 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:17.685567 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:17.685615 (+ 48us) client_negotiation.cc:769] Sending connection context
0924 21:19:17.685688 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 21:19:17.685783 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:19:17.685892 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:17.678044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38392 (local address 127.20.115.252:33257)
0924 21:19:17.678088 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:17.678090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:17.678102 (+ 12us) server_negotiation.cc:407] Connection header received
0924 21:19:17.678127 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:17.678127 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:17.678137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:17.678196 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:17.678541 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.679294 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.680281 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:17.680592 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:17.680666 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:17.683080 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:17.683081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:17.683082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:17.683089 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:17.684674 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.685083 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.685084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.685084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.685142 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:17.685358 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:17.685359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:17.685359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:17.685490 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:17.685571 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:17.685725 (+ 154us) server_negotiation.cc:299] Negotiation successful
0924 21:19:17.685798 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:19:17.688024 1908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b: Bootstrap starting.
I20260924 21:19:17.688750 1908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:17.689107 1908 log.cc:824] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:17.689973 1908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b: No bootstrap required, opened a new log
I20260924 21:19:17.692258 1908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } }
I20260924 21:19:17.692404 1908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:17.692457 1908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11286e1302a64513a14c8467f181f51b, State: Initialized, Role: FOLLOWER
I20260924 21:19:17.692571 1908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } }
I20260924 21:19:17.692906 1908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [sys.catalog]: configured and running, proceeding with master startup.
W20260924 21:19:17.694710 2036 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:19:17.694811 2036 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:19:17.692863 2025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:17.695964 2025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:19:17.696883 1840 master_runner.cc:263] No leader master found from master 11286e1302a64513a14c8467f181f51b
I20260924 21:19:17.696934 1840 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260924 21:19:17.784711 1786 tablet.cc:2369] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:19:17.842571 1936 tablet.cc:2369] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:19:17.937026 1861 tablet.cc:2369] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:19:18.000612 2025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:18.000761 2025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } }
I20260924 21:19:18.001219 2025 leader_election.cc:288] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f791a4748c7477e98829b84846a8993 (127.20.115.254:46497), 7d0048bbb9134a38b0ec9a2dfdf5f27e (127.20.115.252:33257)
I20260924 21:19:18.001541 1957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11286e1302a64513a14c8467f181f51b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" is_pre_election: true
I20260924 21:19:18.001554 1807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11286e1302a64513a14c8467f181f51b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f791a4748c7477e98829b84846a8993" is_pre_election: true
I20260924 21:19:18.001719 1807 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11286e1302a64513a14c8467f181f51b in term 0.
I20260924 21:19:18.001713 1957 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11286e1302a64513a14c8467f181f51b in term 0.
I20260924 21:19:18.001935 1859 leader_election.cc:302] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11286e1302a64513a14c8467f181f51b, 7d0048bbb9134a38b0ec9a2dfdf5f27e; no voters:
I20260924 21:19:18.002100 2025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:19:18.002161 2025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:19:18.002183 2025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:18.003135 2025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } }
I20260924 21:19:18.003278 2025 leader_election.cc:288] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [CANDIDATE]: Term 1 election: Requested vote from peers 9f791a4748c7477e98829b84846a8993 (127.20.115.254:46497), 7d0048bbb9134a38b0ec9a2dfdf5f27e (127.20.115.252:33257)
I20260924 21:19:18.003470 1807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11286e1302a64513a14c8467f181f51b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f791a4748c7477e98829b84846a8993"
I20260924 21:19:18.003499 1957 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11286e1302a64513a14c8467f181f51b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e"
I20260924 21:19:18.003585 1807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:18.003597 1957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:18.004469 1957 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11286e1302a64513a14c8467f181f51b in term 1.
I20260924 21:19:18.004496 1807 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11286e1302a64513a14c8467f181f51b in term 1.
I20260924 21:19:18.004685 1859 leader_election.cc:302] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11286e1302a64513a14c8467f181f51b, 7d0048bbb9134a38b0ec9a2dfdf5f27e; no voters:
I20260924 21:19:18.004807 2025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:19:18.005057 2025 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [term 1 LEADER]: Becoming Leader. State: Replica: 11286e1302a64513a14c8467f181f51b, State: Running, Role: LEADER
I20260924 21:19:18.005213 2025 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } }
I20260924 21:19:18.005649 2038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11286e1302a64513a14c8467f181f51b. Latest consensus state: current_term: 1 leader_uuid: "11286e1302a64513a14c8467f181f51b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:18.005752 2038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:18.005960 2040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:19:18.006152 2040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:19:18.007594 1807 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [term 1 FOLLOWER]: Refusing update from remote peer 11286e1302a64513a14c8467f181f51b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:18.007670 1957 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [term 1 FOLLOWER]: Refusing update from remote peer 11286e1302a64513a14c8467f181f51b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:18.007963 2038 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:18.008087 2025 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:18.009318 2011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11286e1302a64513a14c8467f181f51b. Latest consensus state: current_term: 1 leader_uuid: "11286e1302a64513a14c8467f181f51b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:18.009368 1997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11286e1302a64513a14c8467f181f51b. Latest consensus state: current_term: 1 leader_uuid: "11286e1302a64513a14c8467f181f51b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:18.009434 2011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:19:18.009454 1997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:19:18.010597 2038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11286e1302a64513a14c8467f181f51b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:18.010720 2038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:18.010617 2025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11286e1302a64513a14c8467f181f51b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:18.010789 2042 mvcc.cc:205] Tried to move back new op lower bound from 7333006368796479488 to 7333006368790175744. Current Snapshot: MvccSnapshot[applied={T|T < 7333006368796479488}]
I20260924 21:19:18.010809 2011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11286e1302a64513a14c8467f181f51b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:18.010888 2011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:19:18.010869 1997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11286e1302a64513a14c8467f181f51b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f791a4748c7477e98829b84846a8993" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46497 } } peers { permanent_uuid: "11286e1302a64513a14c8467f181f51b" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 45293 } } peers { permanent_uuid: "7d0048bbb9134a38b0ec9a2dfdf5f27e" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 33257 } } }
I20260924 21:19:18.010943 1997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:19:18.010998 2025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:18.011041 2043 mvcc.cc:205] Tried to move back new op lower bound from 7333006368796479488 to 7333006368790175744. Current Snapshot: MvccSnapshot[applied={T|T < 7333006368796479488}]
I20260924 21:19:18.011533 2040 catalog_manager.cc:1347] Generated new cluster ID: e5e566521f14481fa61a9e2b07544d1c
I20260924 21:19:18.011584 2040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:19:18.028170 2040 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:19:18.028856 2040 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:19:18.032814 2040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11286e1302a64513a14c8467f181f51b: Generated new TSK 0
I20260924 21:19:18.033032 2040 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:19:18.634153 2008 catalog_manager.cc:1259] Loaded cluster ID: e5e566521f14481fa61a9e2b07544d1c
I20260924 21:19:18.634228 2008 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e: loading cluster ID for follower catalog manager: success
I20260924 21:19:18.635025 2008 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e: acquiring CA information for follower catalog manager: success
I20260924 21:19:18.635272 2008 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7d0048bbb9134a38b0ec9a2dfdf5f27e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:19:18.649997 2022 catalog_manager.cc:1259] Loaded cluster ID: e5e566521f14481fa61a9e2b07544d1c
I20260924 21:19:18.650067 2022 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993: loading cluster ID for follower catalog manager: success
I20260924 21:19:18.650902 2022 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993: acquiring CA information for follower catalog manager: success
I20260924 21:19:18.651207 2022 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f791a4748c7477e98829b84846a8993: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:19:19.083317 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 1765
I20260924 21:19:19.087881 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 1840
I20260924 21:19:19.093182 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 1915
2026-09-24T21:19:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3080 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:19:19 dist-test-slave-cxpd krb5kdc[2056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:19:19 dist-test-slave-cxpd krb5kdc[2056](info): set up 2 sockets
Sep 24 21:19:19 dist-test-slave-cxpd krb5kdc[2056](info): commencing operation
krb5kdc: starting...
W20260924 21:19:20.213029 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:19:20 dist-test-slave-cxpd krb5kdc[2056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:19:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:19:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:19:20.328130 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:36175
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:36175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:20.407548 2069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:20.407805 2069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:20.407846 2069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:20.409066 2069 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:20.409135 2069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:20.409159 2069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:20.409188 2069 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:20.409216 2069 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:20.410656 2069 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:36175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:36175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:20.410923 2069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:20.411135 2069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:20.415812 2074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:20.415846 2077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:20.415818 2075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:20.416124 2069 server_base.cc:1034] running on GCE node
I20260924 21:19:20.416412 2069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:20.416700 2069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:20.417863 2069 hybrid_clock.cc:648] HybridClock initialized: now 1790284760417844 us; error 31 us; skew 500 ppm
Sep 24 21:19:20 dist-test-slave-cxpd krb5kdc[2056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284760, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:20.420254 2069 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:19:20.420723 2069 webserver.cc:460] Webserver started at http://127.20.115.254:41351/ using document root <none> and password file <none>
I20260924 21:19:20.420971 2069 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:20.421038 2069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:20.421192 2069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:20.422199 2069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "76670e05fbe64ab0804e2a0106b01cea"
format_stamp: "Formatted at 2026-09-24 21:19:20 on dist-test-slave-cxpd"
server_key: "34e23be2dcd2da1d6bb5dc501f865ecb"
server_key_iv: "7c4d65b687e8be4c735af951aa7db1fa"
server_key_version: "encryptionkey@0"
I20260924 21:19:20.422549 2069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "76670e05fbe64ab0804e2a0106b01cea"
format_stamp: "Formatted at 2026-09-24 21:19:20 on dist-test-slave-cxpd"
server_key: "34e23be2dcd2da1d6bb5dc501f865ecb"
server_key_iv: "7c4d65b687e8be4c735af951aa7db1fa"
server_key_version: "encryptionkey@0"
I20260924 21:19:20.423831 2069 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:19:20.424607 2084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:20.424845 2069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:20.424953 2069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "76670e05fbe64ab0804e2a0106b01cea"
format_stamp: "Formatted at 2026-09-24 21:19:20 on dist-test-slave-cxpd"
server_key: "34e23be2dcd2da1d6bb5dc501f865ecb"
server_key_iv: "7c4d65b687e8be4c735af951aa7db1fa"
server_key_version: "encryptionkey@0"
I20260924 21:19:20.425040 2069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:20.433457 2069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:20.435180 2069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:20.435336 2069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:20.439095 2069 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:36175
I20260924 21:19:20.439122 2136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:36175 every 8 connection(s)
I20260924 21:19:20.439455 2069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:19:20.440272 2137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:20.442282 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 2069
I20260924 21:19:20.442410 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:19:20.442600 2137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea: Bootstrap starting.
I20260924 21:19:20.442615 20943 external_mini_cluster.cc:1428] Setting key 1ec811c8f6f8f037419ff67a35ac74e1
I20260924 21:19:20.443279 2137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:20.443593 2137 log.cc:824] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:20.444398 2137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea: No bootstrap required, opened a new log
I20260924 21:19:20.446579 2137 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76670e05fbe64ab0804e2a0106b01cea" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36175 } }
I20260924 21:19:20.446738 2137 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:20.446770 2137 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76670e05fbe64ab0804e2a0106b01cea, State: Initialized, Role: FOLLOWER
I20260924 21:19:20.446905 2137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76670e05fbe64ab0804e2a0106b01cea" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36175 } }
I20260924 21:19:20.447017 2137 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:19:20.447062 2137 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:19:20.447110 2137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:20.447695 2137 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76670e05fbe64ab0804e2a0106b01cea" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36175 } }
I20260924 21:19:20.447805 2137 leader_election.cc:302] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76670e05fbe64ab0804e2a0106b01cea; no voters:
I20260924 21:19:20.447995 2137 leader_election.cc:288] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:19:20.448065 2142 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:19:20.448256 2137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:19:20.448271 2142 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [term 1 LEADER]: Becoming Leader. State: Replica: 76670e05fbe64ab0804e2a0106b01cea, State: Running, Role: LEADER
I20260924 21:19:20.448560 2142 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76670e05fbe64ab0804e2a0106b01cea" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36175 } }
I20260924 21:19:20.448976 2144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76670e05fbe64ab0804e2a0106b01cea. Latest consensus state: current_term: 1 leader_uuid: "76670e05fbe64ab0804e2a0106b01cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76670e05fbe64ab0804e2a0106b01cea" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36175 } } }
I20260924 21:19:20.449292 2143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76670e05fbe64ab0804e2a0106b01cea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76670e05fbe64ab0804e2a0106b01cea" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36175 } } }
I20260924 21:19:20.449373 2144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:20.449416 2143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea [sys.catalog]: This master's current role is: LEADER
Sep 24 21:19:20 dist-test-slave-cxpd krb5kdc[2056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
W20260924 21:19:20.450289 2157 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:19:20.450398 2157 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:19:20.450872 2153 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:19:20.451406 2153 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:19:20.453835 2140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:20.443109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42648 (local address 127.20.115.254:36175)
0924 21:19:20.443265 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:20.443280 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:20.443303 (+ 23us) server_negotiation.cc:407] Connection header received
0924 21:19:20.443857 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:20.443866 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:20.443926 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:20.444095 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:20.444882 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:20.445738 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:20.446863 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:20.447416 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:20.447504 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:20.450582 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:20.450586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:20.450586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:20.450597 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:20.452184 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:20.452599 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:20.452602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:20.452602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:20.452711 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:20.452955 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:20.452956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:20.452957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:20.453254 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:20.453431 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:20.453533 (+ 102us) server_negotiation.cc:299] Negotiation successful
0924 21:19:20.453665 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260924 21:19:20.454715 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:19:20.457196 2153 catalog_manager.cc:1347] Generated new cluster ID: 14875c490ca24ba39c6f114ecff4700e
I20260924 21:19:20.457242 2153 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:19:20.469760 2153 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:19:20.470495 2153 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:19:20.484802 2153 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76670e05fbe64ab0804e2a0106b01cea: Generated new TSK 0
I20260924 21:19:20.484979 2153 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:19:20.508538 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 2069
2026-09-24T21:19:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1413 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:19:20 dist-test-slave-cxpd krb5kdc[2164](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:19:20 dist-test-slave-cxpd krb5kdc[2164](info): set up 2 sockets
Sep 24 21:19:20 dist-test-slave-cxpd krb5kdc[2164](info): commencing operation
krb5kdc: starting...
W20260924 21:19:21.540905 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:19:21 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:19:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:19:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:19:21.655428 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33809
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:33809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:21.733561 2177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:21.733798 2177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:21.733832 2177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:21.734969 2177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:21.735047 2177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:21.735075 2177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:21.735107 2177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:21.735131 2177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:21.736545 2177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:33809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2177
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:21.736824 2177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:21.737053 2177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:21.741724 2182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:21.741758 2185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:21.741847 2177 server_base.cc:1034] running on GCE node
W20260924 21:19:21.741739 2183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:21.742208 2177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:21.742542 2177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:21.743690 2177 hybrid_clock.cc:648] HybridClock initialized: now 1790284761743669 us; error 33 us; skew 500 ppm
Sep 24 21:19:21 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284761, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:21.746270 2177 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:19:21.746760 2177 webserver.cc:460] Webserver started at http://127.20.115.254:36179/ using document root <none> and password file <none>
I20260924 21:19:21.746980 2177 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:21.747046 2177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:21.747185 2177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:21.748188 2177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "a48862bb0f3749709affa358e2a028d3"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "f8c0d041aad2ce6569a2553d3ceef6a8"
server_key_iv: "76e03ca7429a5d7191d25cfaede23552"
server_key_version: "encryptionkey@0"
I20260924 21:19:21.748566 2177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "a48862bb0f3749709affa358e2a028d3"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "f8c0d041aad2ce6569a2553d3ceef6a8"
server_key_iv: "76e03ca7429a5d7191d25cfaede23552"
server_key_version: "encryptionkey@0"
I20260924 21:19:21.749928 2177 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:19:21.750721 2192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:21.750905 2177 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:21.750999 2177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "a48862bb0f3749709affa358e2a028d3"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "f8c0d041aad2ce6569a2553d3ceef6a8"
server_key_iv: "76e03ca7429a5d7191d25cfaede23552"
server_key_version: "encryptionkey@0"
I20260924 21:19:21.751091 2177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:21.765230 2177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:21.767000 2177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:21.767164 2177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:21.771227 2177 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:33809
I20260924 21:19:21.771255 2244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:33809 every 8 connection(s)
I20260924 21:19:21.771597 2177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:19:21.772534 2245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:21.774686 2245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Bootstrap starting.
I20260924 21:19:21.775202 2245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:21.775426 2245 log.cc:824] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:21.776013 2245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: No bootstrap required, opened a new log
I20260924 21:19:21.777590 2245 raft_consensus.cc:348] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:21.777719 2245 raft_consensus.cc:374] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:21.777741 2245 raft_consensus.cc:729] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48862bb0f3749709affa358e2a028d3, State: Initialized, Role: FOLLOWER
I20260924 21:19:21.777834 2245 consensus_queue.cc:260] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:21.777885 2245 raft_consensus.cc:388] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:19:21.777905 2245 raft_consensus.cc:482] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:19:21.777971 2245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:21.778755 2245 raft_consensus.cc:504] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:21.778864 2245 leader_election.cc:302] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a48862bb0f3749709affa358e2a028d3; no voters:
I20260924 21:19:21.779031 2245 leader_election.cc:288] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:19:21.779070 2248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:19:21.779238 2248 raft_consensus.cc:686] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 1 LEADER]: Becoming Leader. State: Replica: a48862bb0f3749709affa358e2a028d3, State: Running, Role: LEADER
I20260924 21:19:21.779286 2245 sys_catalog.cc:564] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:19:21.779378 2248 consensus_queue.cc:237] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:21.779595 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 2177
I20260924 21:19:21.779692 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:19:21.779749 2249 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a48862bb0f3749709affa358e2a028d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } } }
I20260924 21:19:21.779829 2249 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:21.779917 20943 external_mini_cluster.cc:1428] Setting key d2eafa6b80f8e44f43887f1716c4dc82
I20260924 21:19:21.779995 2250 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a48862bb0f3749709affa358e2a028d3. Latest consensus state: current_term: 1 leader_uuid: "a48862bb0f3749709affa358e2a028d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } } }
I20260924 21:19:21.780028 2256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:19:21.780062 2250 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:21.780499 2256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:19:21.782152 2256 catalog_manager.cc:1347] Generated new cluster ID: 11906873e8524c428caf319a1cd9e79c
I20260924 21:19:21.782202 2256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:19:21 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284761, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:21.792162 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:21.780509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58736 (local address 127.20.115.254:33809)
0924 21:19:21.780842 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:21.780852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:21.780932 (+ 80us) server_negotiation.cc:407] Connection header received
0924 21:19:21.781289 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:21.781298 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:21.781357 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:21.781492 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:21.782249 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:21.783618 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:21.784700 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:21.785304 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:21.785401 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:21.788201 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:21.788205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:21.788205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:21.788217 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:21.790602 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:21.790974 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:21.790977 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:21.790977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:21.791097 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:21.791292 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:21.791293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:21.791293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:21.791591 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:21.791742 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:21.791813 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 21:19:21.791943 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":77,"threads_started":1}
I20260924 21:19:21.794528 2256 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:19:21.795202 2256 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:19:21.808923 2256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Generated new TSK 0
I20260924 21:19:21.809104 2256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:19:21.840006 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:33809
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:21.923760 2271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:21.923993 2271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:21.924033 2271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:21.925292 2271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:21.925362 2271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:21.925407 2271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:19:21.927003 2271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:21.927291 2271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:21.927524 2271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:21.932976 2277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:21.932986 2279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:21.932976 2276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:21.933285 2271 server_base.cc:1034] running on GCE node
I20260924 21:19:21.933468 2271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:21.933718 2271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:21.934860 2271 hybrid_clock.cc:648] HybridClock initialized: now 1790284761934844 us; error 39 us; skew 500 ppm
Sep 24 21:19:21 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284761, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:21.937402 2271 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:19:21.937853 2271 webserver.cc:460] Webserver started at http://127.20.115.193:45009/ using document root <none> and password file <none>
I20260924 21:19:21.938046 2271 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:21.938120 2271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:21.938257 2271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:21.939263 2271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "bd62766ef9474d1d9c1d51729d9aea66"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "3938e2974adf449fe028a203e0b0ac27"
server_key_iv: "ea968c02126d7e7ea658ba8d027c3aec"
server_key_version: "encryptionkey@0"
I20260924 21:19:21.939551 2271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "bd62766ef9474d1d9c1d51729d9aea66"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "3938e2974adf449fe028a203e0b0ac27"
server_key_iv: "ea968c02126d7e7ea658ba8d027c3aec"
server_key_version: "encryptionkey@0"
I20260924 21:19:21.940762 2271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260924 21:19:21.941495 2286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:21.941650 2271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:19:21.941735 2271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "bd62766ef9474d1d9c1d51729d9aea66"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "3938e2974adf449fe028a203e0b0ac27"
server_key_iv: "ea968c02126d7e7ea658ba8d027c3aec"
server_key_version: "encryptionkey@0"
I20260924 21:19:21.941787 2271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:21.958977 2271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:21.960825 2271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:21.960935 2271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:21.961102 2271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:21.961406 2271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:21.961440 2271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:21.961462 2271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:21.961506 2271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:21.967058 2271 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:45835
I20260924 21:19:21.967159 2399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:45835 every 8 connection(s)
I20260924 21:19:21.967444 2271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:19:21 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284761, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:21.975112 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 2271
I20260924 21:19:21.975234 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:19:21.975433 20943 external_mini_cluster.cc:1428] Setting key 1312c8bd60f56eb5ca028829ca9a860d
I20260924 21:19:21.976830 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:21.968155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:42971 (local address 127.20.115.254:33809)
0924 21:19:21.968201 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:21.968203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:21.968412 (+ 209us) server_negotiation.cc:407] Connection header received
0924 21:19:21.968910 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:21.968912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:21.968929 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:21.968999 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:21.969880 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:21.970385 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:21.971323 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:21.971516 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:21.971574 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:21.973712 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:21.973713 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:21.973713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:21.973720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:21.975241 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:21.975725 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:21.975726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:21.975726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:21.975786 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:21.976103 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:21.976104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:21.976104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:21.976253 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:21.976328 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:21.976576 (+ 248us) server_negotiation.cc:299] Negotiation successful
0924 21:19:21.976671 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:19:21.976862 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:21.968143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:42971)
0924 21:19:21.968286 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:21.968320 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:21.968824 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:21.969048 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:21.969051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:21.969594 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:21.969779 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:21.969782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:21.970466 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:21.970466 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:21.971251 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:21.971253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:21.971599 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:21.971600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:21.971697 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:21.972209 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:21.972217 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:21.973627 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:21.975314 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:21.975321 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:21.975323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:21.975623 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:21.975868 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:21.975869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:21.975869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:21.976032 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:21.976324 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:21.976329 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:21.976439 (+ 110us) client_negotiation.cc:769] Sending connection context
0924 21:19:21.976525 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:19:21.976650 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":86,"threads_started":1}
I20260924 21:19:21.977310 2400 heartbeater.cc:347] Connected to a master server at 127.20.115.254:33809
I20260924 21:19:21.977408 2400 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:21.977617 2400 heartbeater.cc:511] Master 127.20.115.254:33809 requested a full tablet report, sending...
I20260924 21:19:21.978164 2208 ts_manager.cc:194] Registered new tserver with Master: bd62766ef9474d1d9c1d51729d9aea66 (127.20.115.193:45835)
I20260924 21:19:21.978766 2208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:42971
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:19:22.001639 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:33809
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:22.080127 2405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:22.080315 2405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:22.080338 2405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:22.081557 2405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:22.081601 2405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:22.081629 2405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:19:22.083021 2405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2405
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:22.083251 2405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:22.083475 2405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:22.088682 2411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:22.088711 2413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:22.088881 2405 server_base.cc:1034] running on GCE node
W20260924 21:19:22.088913 2410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:22.089143 2405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:22.089395 2405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:22.090579 2405 hybrid_clock.cc:648] HybridClock initialized: now 1790284762090556 us; error 33 us; skew 500 ppm
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:22.093014 2405 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:19:22.093478 2405 webserver.cc:460] Webserver started at http://127.20.115.194:38963/ using document root <none> and password file <none>
I20260924 21:19:22.093703 2405 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:22.093775 2405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:22.093909 2405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:22.095023 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "75013a6d236b4f36a1d8bf56aacf76e8"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "5dea4c45ccc61d7ca7dde152bc85a306"
server_key_iv: "6dc0fc3e6c4e231045fa0ecefd507b5a"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.095382 2405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "75013a6d236b4f36a1d8bf56aacf76e8"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "5dea4c45ccc61d7ca7dde152bc85a306"
server_key_iv: "6dc0fc3e6c4e231045fa0ecefd507b5a"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.096686 2405 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:19:22.097500 2420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.097668 2405 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:19:22.097765 2405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "75013a6d236b4f36a1d8bf56aacf76e8"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "5dea4c45ccc61d7ca7dde152bc85a306"
server_key_iv: "6dc0fc3e6c4e231045fa0ecefd507b5a"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.097849 2405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:22.116689 2405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:22.119143 2405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:22.119366 2405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:22.119647 2405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:22.119968 2405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:22.120029 2405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.120070 2405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:22.120100 2405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.126159 2405 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:39767
I20260924 21:19:22.126187 2533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:39767 every 8 connection(s)
I20260924 21:19:22.126523 2405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:22.135191 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.127326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:47595 (local address 127.20.115.254:33809)
0924 21:19:22.127389 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.127391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.127591 (+ 200us) server_negotiation.cc:407] Connection header received
0924 21:19:22.128054 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.128055 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.128065 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.128130 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:22.129138 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.129638 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.130588 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.130789 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.130847 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.132912 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:22.132913 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:22.132914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:22.132919 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:22.134100 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.134445 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.134445 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.134446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.134491 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.134693 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.134694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.134694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.134785 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:22.134835 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.135068 (+ 233us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.135129 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:19:22.135416 2536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.127271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:47595)
0924 21:19:22.127437 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.127481 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.127964 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.128198 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.128201 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.128834 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:22.129053 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.129058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.129714 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.129715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.130490 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.130492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.130863 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.130864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.130963 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.131501 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:22.131509 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:22.132826 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:22.134174 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.134180 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.134182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.134381 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.134549 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.134549 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.134550 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.134641 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.134837 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:22.134841 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:22.134962 (+ 121us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.135048 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.135163 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260924 21:19:22.135911 2534 heartbeater.cc:347] Connected to a master server at 127.20.115.254:33809
I20260924 21:19:22.136013 2534 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:22.136184 2534 heartbeater.cc:511] Master 127.20.115.254:33809 requested a full tablet report, sending...
I20260924 21:19:22.136368 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 2405
I20260924 21:19:22.136469 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:19:22.136574 2208 ts_manager.cc:194] Registered new tserver with Master: 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767)
I20260924 21:19:22.136682 20943 external_mini_cluster.cc:1428] Setting key 77c0666fe6ec37568df7cb7896af892c
I20260924 21:19:22.137235 2208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:47595
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:19:22.160383 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:33809
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:22.239717 2539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:22.239959 2539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:22.239995 2539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:22.241133 2539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:22.241204 2539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:22.241250 2539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:19:22.242666 2539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:22.242937 2539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:22.243162 2539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:22.248181 2544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:22.248174 2547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:22.248174 2545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:22.248324 2539 server_base.cc:1034] running on GCE node
I20260924 21:19:22.248646 2539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:22.248927 2539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:22.250095 2539 hybrid_clock.cc:648] HybridClock initialized: now 1790284762250097 us; error 47 us; skew 500 ppm
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:22.252626 2539 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:19:22.253165 2539 webserver.cc:460] Webserver started at http://127.20.115.195:34781/ using document root <none> and password file <none>
I20260924 21:19:22.253396 2539 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:22.253463 2539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:22.253605 2539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:22.254671 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "d40f314fcd3f461ab1039758e46aa2d1"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "eff01ee92e6ba67275240b316c1121c1"
server_key_iv: "75f1c9c1d57135524acfdef70d2c3d90"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.255064 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "d40f314fcd3f461ab1039758e46aa2d1"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "eff01ee92e6ba67275240b316c1121c1"
server_key_iv: "75f1c9c1d57135524acfdef70d2c3d90"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.256343 2539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:19:22.257138 2554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.257303 2539 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:19:22.257406 2539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "d40f314fcd3f461ab1039758e46aa2d1"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "eff01ee92e6ba67275240b316c1121c1"
server_key_iv: "75f1c9c1d57135524acfdef70d2c3d90"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.257480 2539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:22.274680 2539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:22.277087 2539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:22.277252 2539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:22.277488 2539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:22.277803 2539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:22.277858 2539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.277901 2539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:22.277928 2539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.284003 2539 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:38527
I20260924 21:19:22.284049 2667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:38527 every 8 connection(s)
I20260924 21:19:22.284396 2539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:19:22.285233 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 2539
I20260924 21:19:22.285355 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:19:22.285526 20943 external_mini_cluster.cc:1428] Setting key c5da34c304418c585f0e211b463b0beb
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:22.293664 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.285142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:51369 (local address 127.20.115.254:33809)
0924 21:19:22.285192 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.285194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.285378 (+ 184us) server_negotiation.cc:407] Connection header received
0924 21:19:22.285966 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.285968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.285980 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.286044 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:22.287067 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.287571 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.288589 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.288837 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.288930 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.291058 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:22.291059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:22.291059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:22.291068 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:22.292268 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.292675 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.292676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.292677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.292739 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.293013 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.293014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.293015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.293139 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:22.293220 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.293490 (+ 270us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.293576 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:19:22.293851 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.285101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:51369)
0924 21:19:22.285259 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.285303 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.285859 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.286117 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.286121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.286676 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:22.286952 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.286957 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.287672 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.287673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.288476 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.288478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.288940 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.288942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.289079 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.289564 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:22.289573 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:22.290939 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:22.292367 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.292373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.292375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.292595 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.292835 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.292836 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.292836 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.292945 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.293210 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:22.293217 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:22.293356 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.293466 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.293617 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260924 21:19:22.294344 2668 heartbeater.cc:347] Connected to a master server at 127.20.115.254:33809
I20260924 21:19:22.294430 2668 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:22.294603 2668 heartbeater.cc:511] Master 127.20.115.254:33809 requested a full tablet report, sending...
I20260924 21:19:22.295125 2209 ts_manager.cc:194] Registered new tserver with Master: d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527)
I20260924 21:19:22.295495 2209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:51369
I20260924 21:19:22.297150 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:19:22.303862 2266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.298440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58744 (local address 127.20.115.254:33809)
0924 21:19:22.298503 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.298505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.298589 (+ 84us) server_negotiation.cc:407] Connection header received
0924 21:19:22.298626 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.298626 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.298634 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.298683 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:22.299201 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.299653 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.300384 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.300592 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.300649 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.301825 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:22.301826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:22.301826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:22.301832 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:22.302997 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.303294 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.303295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.303295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.303344 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.303466 (+ 122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.303466 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.303467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.303592 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:22.303663 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.303751 (+ 88us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.303805 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:19:22.305539 2209 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58744:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:19:22.307076 2209 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:19:22.313808 2680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.309641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.20.115.194:39767)
0924 21:19:22.310022 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.310025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.310035 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:19:22.310072 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.310074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.310088 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.310192 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:22.310295 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.311429 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.312719 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.313430 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.313511 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.313575 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.313711 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.313744 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":83,"threads_started":1}
I20260924 21:19:22.313814 2676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.309457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:39767 (local address 127.0.0.1:35138)
0924 21:19:22.309647 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.309680 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.309741 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.310192 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.310193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.310195 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:22.310234 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.310236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.311546 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.311547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.312640 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.312642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.313519 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.313520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.313599 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.313614 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.313688 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.313740 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":3,"thread_start_us":60,"threads_started":1}
I20260924 21:19:22.314505 2677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.309387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45835 (local address 127.0.0.1:56170)
0924 21:19:22.309566 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.309787 (+ 221us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.309829 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.310102 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.310105 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.310107 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:22.310178 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.310181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.311229 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.311230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.312543 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.312545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.314232 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.314233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.314312 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.314317 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.314381 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.314422 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":39,"thread_start_us":86,"threads_started":1}
I20260924 21:19:22.314725 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.309624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:38527 (local address 127.0.0.1:49028)
0924 21:19:22.309710 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.309720 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.309774 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.310203 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.310204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.310207 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:22.310275 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.310278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.312725 (+ 2447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.312726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.313490 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.313492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.314470 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.314471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.314543 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.314547 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.314600 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.314642 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72,"mutex_wait_us":2,"thread_start_us":53,"threads_started":1}
I20260924 21:19:22.314643 2468 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8865fa3ba4476cb309fcc4c8241f16 (DEFAULT_TABLE table=test-table [id=d06d43db62aa4d6bba6b40be9178825d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:22.314775 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.309478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.20.115.193:45835)
0924 21:19:22.309950 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.309954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.309964 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:19:22.309995 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.309997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.310009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.310098 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:22.310264 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.311120 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.312625 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.313428 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.313511 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.313570 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.314411 (+ 841us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.314458 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":84,"threads_started":1}
I20260924 21:19:22.314997 2681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.309798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49028 (local address 127.20.115.195:38527)
0924 21:19:22.309963 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.309968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.309979 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:19:22.310017 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.310020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.310036 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.310126 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:22.310358 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.311428 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.313587 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.314329 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.314392 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.314439 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.314824 (+ 385us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.314879 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":103,"threads_started":1}
I20260924 21:19:22.315040 2468 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8865fa3ba4476cb309fcc4c8241f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:22.315404 2334 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8865fa3ba4476cb309fcc4c8241f16 (DEFAULT_TABLE table=test-table [id=d06d43db62aa4d6bba6b40be9178825d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:22.315707 2334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8865fa3ba4476cb309fcc4c8241f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:22.318572 2682 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Bootstrap starting.
I20260924 21:19:22.319114 2683 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Bootstrap starting.
I20260924 21:19:22.319232 2682 tablet_bootstrap.cc:654] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:22.319245 2602 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8865fa3ba4476cb309fcc4c8241f16 (DEFAULT_TABLE table=test-table [id=d06d43db62aa4d6bba6b40be9178825d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:22.319588 2602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8865fa3ba4476cb309fcc4c8241f16. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:22.319770 2683 tablet_bootstrap.cc:654] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:22.319864 2682 log.cc:824] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:22.320390 2683 log.cc:824] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:22.320712 2682 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: No bootstrap required, opened a new log
I20260924 21:19:22.320878 2682 ts_tablet_manager.cc:1397] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:19:22.321130 2683 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: No bootstrap required, opened a new log
I20260924 21:19:22.321206 2683 ts_tablet_manager.cc:1397] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:19:22.322721 2683 raft_consensus.cc:348] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.322875 2683 raft_consensus.cc:374] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:22.322939 2683 raft_consensus.cc:729] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62766ef9474d1d9c1d51729d9aea66, State: Initialized, Role: FOLLOWER
I20260924 21:19:22.322970 2682 raft_consensus.cc:348] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.323145 2682 raft_consensus.cc:374] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:22.323194 2682 raft_consensus.cc:729] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75013a6d236b4f36a1d8bf56aacf76e8, State: Initialized, Role: FOLLOWER
I20260924 21:19:22.323184 2683 consensus_queue.cc:260] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.323314 2682 consensus_queue.cc:260] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.323480 2683 ts_tablet_manager.cc:1428] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:19:22.323505 2400 heartbeater.cc:503] Master 127.20.115.254:33809 was elected leader, sending a full tablet report...
I20260924 21:19:22.323671 2686 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: Bootstrap starting.
I20260924 21:19:22.323920 2534 heartbeater.cc:503] Master 127.20.115.254:33809 was elected leader, sending a full tablet report...
I20260924 21:19:22.324150 2682 ts_tablet_manager.cc:1428] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:19:22.324311 2686 tablet_bootstrap.cc:654] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:22.324914 2686 log.cc:824] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:22.325644 2686 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: No bootstrap required, opened a new log
I20260924 21:19:22.325699 2686 ts_tablet_manager.cc:1397] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:19:22.327049 2686 raft_consensus.cc:348] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.327164 2686 raft_consensus.cc:374] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:22.327185 2686 raft_consensus.cc:729] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40f314fcd3f461ab1039758e46aa2d1, State: Initialized, Role: FOLLOWER
I20260924 21:19:22.327312 2686 consensus_queue.cc:260] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.327579 2686 ts_tablet_manager.cc:1428] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:19:22.327605 2668 heartbeater.cc:503] Master 127.20.115.254:33809 was elected leader, sending a full tablet report...
W20260924 21:19:22.377149 2535 tablet.cc:2369] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:19:22.380425 2687 raft_consensus.cc:482] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:22.380564 2687 raft_consensus.cc:504] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.380980 2687 leader_election.cc:288] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767), d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527)
I20260924 21:19:22.385126 2680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.381230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:36739 (local address 127.20.115.194:39767)
0924 21:19:22.381276 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.381279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.381290 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:19:22.381391 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.381391 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.381401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.381454 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:22.381698 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.382501 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.384094 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.384675 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.384757 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.384765 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.385020 (+ 255us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.385061 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:19:22.385126 2681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.381295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:32877 (local address 127.20.115.195:38527)
0924 21:19:22.381338 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.381339 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.381555 (+ 216us) server_negotiation.cc:407] Connection header received
0924 21:19:22.381590 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.381591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.381599 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.381636 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:22.381797 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.382507 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.384095 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.384675 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.384756 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.384765 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.385021 (+ 256us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.385058 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:19:22.385133 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.381165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:39767 (local address 127.20.115.193:36739)
0924 21:19:22.381233 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.381248 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.381301 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.381505 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.381506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.381508 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:22.381590 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.381592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.382608 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.382609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.383969 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.383973 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.384820 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.384821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.384917 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.384919 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.384988 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.385032 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":112}
I20260924 21:19:22.385139 2691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.381218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:38527 (local address 127.20.115.193:32877)
0924 21:19:22.381476 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.381486 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.381539 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.381665 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.381665 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.381667 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:22.381732 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.381733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.382608 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.382609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.384009 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.384010 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.384819 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.384820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.384911 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.384916 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.385008 (+ 92us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.385038 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":78,"thread_start_us":137,"threads_started":1}
I20260924 21:19:22.385543 2488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8865fa3ba4476cb309fcc4c8241f16" candidate_uuid: "bd62766ef9474d1d9c1d51729d9aea66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" is_pre_election: true
I20260924 21:19:22.385725 2488 raft_consensus.cc:2454] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd62766ef9474d1d9c1d51729d9aea66 in term 0.
I20260924 21:19:22.385715 2622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8865fa3ba4476cb309fcc4c8241f16" candidate_uuid: "bd62766ef9474d1d9c1d51729d9aea66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40f314fcd3f461ab1039758e46aa2d1" is_pre_election: true
I20260924 21:19:22.385838 2622 raft_consensus.cc:2454] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd62766ef9474d1d9c1d51729d9aea66 in term 0.
I20260924 21:19:22.386087 2288 leader_election.cc:302] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd62766ef9474d1d9c1d51729d9aea66, d40f314fcd3f461ab1039758e46aa2d1; no voters:
I20260924 21:19:22.386240 2687 raft_consensus.cc:2781] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:19:22.386317 2687 raft_consensus.cc:482] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:19:22.386348 2687 raft_consensus.cc:3037] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:22.387284 2687 raft_consensus.cc:504] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.387442 2687 leader_election.cc:288] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 1 election: Requested vote from peers 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767), d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527)
I20260924 21:19:22.387598 2622 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8865fa3ba4476cb309fcc4c8241f16" candidate_uuid: "bd62766ef9474d1d9c1d51729d9aea66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40f314fcd3f461ab1039758e46aa2d1"
I20260924 21:19:22.387595 2488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8865fa3ba4476cb309fcc4c8241f16" candidate_uuid: "bd62766ef9474d1d9c1d51729d9aea66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75013a6d236b4f36a1d8bf56aacf76e8"
I20260924 21:19:22.387679 2622 raft_consensus.cc:3037] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:22.387708 2488 raft_consensus.cc:3037] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:22.388444 2622 raft_consensus.cc:2454] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62766ef9474d1d9c1d51729d9aea66 in term 1.
I20260924 21:19:22.388525 2488 raft_consensus.cc:2454] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd62766ef9474d1d9c1d51729d9aea66 in term 1.
I20260924 21:19:22.388607 2288 leader_election.cc:302] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd62766ef9474d1d9c1d51729d9aea66, d40f314fcd3f461ab1039758e46aa2d1; no voters:
I20260924 21:19:22.388682 2687 raft_consensus.cc:2781] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:19:22.388914 2687 raft_consensus.cc:686] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 LEADER]: Becoming Leader. State: Replica: bd62766ef9474d1d9c1d51729d9aea66, State: Running, Role: LEADER
I20260924 21:19:22.389031 2687 consensus_queue.cc:237] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.389966 2209 catalog_manager.cc:5495] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd62766ef9474d1d9c1d51729d9aea66 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "bd62766ef9474d1d9c1d51729d9aea66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } health_report { overall_health: UNKNOWN } } }
I20260924 21:19:22.409464 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.407145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.20.115.193:45835)
0924 21:19:22.407209 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.407211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.407219 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:19:22.407269 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.407270 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.407288 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.407348 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:19:22.407468 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.407961 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.408697 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.408970 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.409026 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.409164 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:19:22.409256 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:19:22.409316 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.409350 (+ 34us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.409372 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:19:22.411264 2488 raft_consensus.cc:1261] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Refusing update from remote peer bd62766ef9474d1d9c1d51729d9aea66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:22.411264 2622 raft_consensus.cc:1261] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [term 1 FOLLOWER]: Refusing update from remote peer bd62766ef9474d1d9c1d51729d9aea66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:22.411671 2692 consensus_queue.cc:1035] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:22.411796 2687 consensus_queue.cc:1035] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:22.414036 2695 mvcc.cc:205] Tried to move back new op lower bound from 7333006386833698816 to 7333006386746085376. Current Snapshot: MvccSnapshot[applied={T|T < 7333006386833698816}]
I20260924 21:19:22.416766 2208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:19:22.416884 2208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58744:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:19:22.417658 2208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Sending DeleteTablet for 3 replicas of tablet 7e8865fa3ba4476cb309fcc4c8241f16
I20260924 21:19:22.418115 2602 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8865fa3ba4476cb309fcc4c8241f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:22 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:49028
I20260924 21:19:22.418120 2468 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8865fa3ba4476cb309fcc4c8241f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:22 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:35138
I20260924 21:19:22.418316 2334 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8865fa3ba4476cb309fcc4c8241f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:22 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:56170
I20260924 21:19:22.418375 2705 tablet_replica.cc:330] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: stopping tablet replica
I20260924 21:19:22.418370 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 2271
I20260924 21:19:22.418468 2705 raft_consensus.cc:2229] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:19:22.418565 2705 raft_consensus.cc:2258] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:22.419265 2705 ts_tablet_manager.cc:1905] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:19:22.421271 2705 ts_tablet_manager.cc:1918] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:19:22.421350 2705 log.cc:1196] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/7e8865fa3ba4476cb309fcc4c8241f16
I20260924 21:19:22.421669 2705 ts_tablet_manager.cc:1939] T 7e8865fa3ba4476cb309fcc4c8241f16 P d40f314fcd3f461ab1039758e46aa2d1: Deleting consensus metadata
I20260924 21:19:22.422243 2194 catalog_manager.cc:4841] TS d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527): tablet 7e8865fa3ba4476cb309fcc4c8241f16 (table test-table [id=d06d43db62aa4d6bba6b40be9178825d]) successfully deleted
W20260924 21:19:22.424674 2194 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:45835 (error 108)
I20260924 21:19:22.424759 2704 tablet_replica.cc:330] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: stopping tablet replica
I20260924 21:19:22.424914 2704 raft_consensus.cc:2229] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:19:22.425037 2704 raft_consensus.cc:2258] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:22.425254 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.425124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45835 (local address 127.0.0.1:57336)
0924 21:19:22.425184 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.425223 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 21:19:22.425415 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 2405
I20260924 21:19:22.425508 2704 ts_tablet_manager.cc:1905] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 21:19:22.429399 2194 catalog_manager.cc:4584] TS bd62766ef9474d1d9c1d51729d9aea66 (127.20.115.193:45835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e8865fa3ba4476cb309fcc4c8241f16: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45835: connect: Connection refused (error 111)
I20260924 21:19:22.430222 2679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.430113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:39767 (local address 127.0.0.1:37774)
0924 21:19:22.430163 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.430191 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.194:39767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 21:19:22.430292 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 2539
I20260924 21:19:22.435129 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 2177
I20260924 21:19:22.440174 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33809
--webserver_interface=127.20.115.254
--webserver_port=36179
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:33809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:22.521570 2708 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:22.521821 2708 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:22.521855 2708 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:22.522992 2708 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:22.523061 2708 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:22.523084 2708 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:22.523106 2708 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:22.523134 2708 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:22.524559 2708 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:33809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2708
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:22.524845 2708 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:22.525071 2708 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:22.533435 2715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:22.533427 2717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:22.533471 2708 server_base.cc:1034] running on GCE node
W20260924 21:19:22.533452 2714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:22.533896 2708 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:22.534229 2708 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:22.535410 2708 hybrid_clock.cc:648] HybridClock initialized: now 1790284762535381 us; error 36 us; skew 500 ppm
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:22.538149 2708 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:19:22.538571 2708 webserver.cc:460] Webserver started at http://127.20.115.254:36179/ using document root <none> and password file <none>
I20260924 21:19:22.538803 2708 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:22.538861 2708 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:22.540205 2708 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:22.540899 2724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.541090 2708 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:22.541188 2708 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "a48862bb0f3749709affa358e2a028d3"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "f8c0d041aad2ce6569a2553d3ceef6a8"
server_key_iv: "76e03ca7429a5d7191d25cfaede23552"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.541582 2708 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:22.566262 2708 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:22.568382 2708 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:22.568543 2708 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:22.572449 2708 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:33809
I20260924 21:19:22.572487 2776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:33809 every 8 connection(s)
I20260924 21:19:22.572953 2708 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:19:22.574466 2777 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:19:22.575138 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 2708
I20260924 21:19:22.575337 2777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Bootstrap starting.
I20260924 21:19:22.575444 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:45835
--local_ip_for_outbound_sockets=127.20.115.193
--tserver_master_addrs=127.20.115.254:33809
--webserver_port=45009
--webserver_interface=127.20.115.193
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:19:22.583969 2777 log.cc:824] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:22.585981 2777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:19:22.586298 2777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Bootstrap complete.
I20260924 21:19:22.588714 2777 raft_consensus.cc:348] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:22.589072 2777 raft_consensus.cc:729] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48862bb0f3749709affa358e2a028d3, State: Initialized, Role: FOLLOWER
I20260924 21:19:22.589243 2777 consensus_queue.cc:260] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:22.589342 2777 raft_consensus.cc:388] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:19:22.589398 2777 raft_consensus.cc:482] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:19:22.589447 2777 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:19:22.590425 2777 raft_consensus.cc:504] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:22.590541 2777 leader_election.cc:302] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a48862bb0f3749709affa358e2a028d3; no voters:
I20260924 21:19:22.590742 2777 leader_election.cc:288] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 21:19:22.590775 2781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:19:22.590878 2781 raft_consensus.cc:686] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 2 LEADER]: Becoming Leader. State: Replica: a48862bb0f3749709affa358e2a028d3, State: Running, Role: LEADER
I20260924 21:19:22.591017 2777 sys_catalog.cc:564] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:19:22.591017 2781 consensus_queue.cc:237] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:22.591419 2783 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a48862bb0f3749709affa358e2a028d3. Latest consensus state: current_term: 2 leader_uuid: "a48862bb0f3749709affa358e2a028d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } } }
I20260924 21:19:22.591562 2783 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:22.591737 2782 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a48862bb0f3749709affa358e2a028d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } } }
I20260924 21:19:22.591791 2782 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:22.592857 2797 catalog_manager.cc:1259] Loaded cluster ID: 11906873e8524c428caf319a1cd9e79c
I20260924 21:19:22.592907 2797 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: loading cluster ID for follower catalog manager: success
W20260924 21:19:22.593094 2797 catalog_manager.cc:1576] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260924 21:19:22.593137 2797 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260924 21:19:22.593169 2791 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:19:22.593511 2791 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:19:22.593577 2791 catalog_manager.cc:1259] Loaded cluster ID: 11906873e8524c428caf319a1cd9e79c
I20260924 21:19:22.593601 2791 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:19:22.593739 2791 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260924 21:19:22.598487 2791 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260924 21:19:22.600901 2791 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:19:22.601083 2791 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Loaded TSK: 0
I20260924 21:19:22.601297 2791 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 21:19:22.657802 2779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:22.658038 2779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:22.658079 2779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:22.659197 2779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:22.659266 2779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:22.659312 2779 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:19:22.660756 2779 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:45835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=45009
--tserver_master_addrs=127.20.115.254:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:22.661043 2779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:22.661266 2779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:22.666282 2807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:22.666455 2779 server_base.cc:1034] running on GCE node
W20260924 21:19:22.666323 2804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:22.666323 2805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:22.666887 2779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:22.667171 2779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:22.668378 2779 hybrid_clock.cc:648] HybridClock initialized: now 1790284762668364 us; error 28 us; skew 500 ppm
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:22.671499 2779 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:19:22.671998 2779 webserver.cc:460] Webserver started at http://127.20.115.193:45009/ using document root <none> and password file <none>
I20260924 21:19:22.672242 2779 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:22.672323 2779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:22.673802 2779 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:22.676056 2814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.676194 2779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260924 21:19:22.676271 2779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "bd62766ef9474d1d9c1d51729d9aea66"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "3938e2974adf449fe028a203e0b0ac27"
server_key_iv: "ea968c02126d7e7ea658ba8d027c3aec"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.676820 2779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:22.694525 2779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:22.697115 2779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:22.697278 2779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:22.697538 2779 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:22.697971 2821 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:19:22.699916 2779 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:19:22.699973 2779 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:19:22.700001 2779 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:19:22.700743 2779 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:19:22.700834 2779 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260924 21:19:22.700894 2821 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Bootstrap starting.
I20260924 21:19:22.708222 2779 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:45835
I20260924 21:19:22.708295 2928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:45835 every 8 connection(s)
I20260924 21:19:22.708698 2779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:19:22.710775 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 2779
I20260924 21:19:22.711086 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:39767
--local_ip_for_outbound_sockets=127.20.115.194
--tserver_master_addrs=127.20.115.254:33809
--webserver_port=38963
--webserver_interface=127.20.115.194
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:19:22.712481 2821 log.cc:824] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:22.715376 2821 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:19:22.715692 2821 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Bootstrap complete.
I20260924 21:19:22.716948 2821 ts_tablet_manager.cc:1397] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:22.718994 2821 raft_consensus.cc:348] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.719404 2821 raft_consensus.cc:729] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62766ef9474d1d9c1d51729d9aea66, State: Initialized, Role: FOLLOWER
I20260924 21:19:22.719590 2821 consensus_queue.cc:260] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.719900 2821 ts_tablet_manager.cc:1428] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260924 21:19:22.722440 2931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.709491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:54379)
0924 21:19:22.709705 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.709757 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.710234 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.711584 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.711588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.712012 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:22.712303 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.712311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.714032 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.714033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.715302 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.715305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.716307 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.716308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.716450 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.717130 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:22.717139 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:22.718760 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:22.720774 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.720807 (+ 33us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.720809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.721064 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.721340 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.721340 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.721341 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.721444 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.721822 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:22.721829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:22.721963 (+ 134us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.722077 (+ 114us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.722248 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260924 21:19:22.722460 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.709592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:54379 (local address 127.20.115.254:33809)
0924 21:19:22.711037 (+ 1445us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.711046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.711093 (+ 47us) server_negotiation.cc:407] Connection header received
0924 21:19:22.711237 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.711243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.711299 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.711432 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:22.712464 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.713880 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.715407 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.716091 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.716195 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.718887 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:22.718892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:22.718892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:22.718903 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:22.720666 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.721154 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.721157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.721157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.721254 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.721510 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.721511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.721511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.721743 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:22.721841 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.722127 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.722248 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1374,"thread_start_us":100,"threads_started":1}
I20260924 21:19:22.723599 2929 heartbeater.cc:347] Connected to a master server at 127.20.115.254:33809
I20260924 21:19:22.723712 2929 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:22.723881 2929 heartbeater.cc:511] Master 127.20.115.254:33809 requested a full tablet report, sending...
I20260924 21:19:22.724598 2741 ts_manager.cc:194] Registered new tserver with Master: bd62766ef9474d1d9c1d51729d9aea66 (127.20.115.193:45835)
I20260924 21:19:22.725699 2741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:54379
I20260924 21:19:22.726475 2929 heartbeater.cc:503] Master 127.20.115.254:33809 was elected leader, sending a full tablet report...
I20260924 21:19:22.731920 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.725293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.20.115.193:45835)
0924 21:19:22.729081 (+ 3788us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.729084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.729091 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:19:22.729115 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.729117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.729128 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.729210 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:22.729350 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.730092 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.731074 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.731514 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.731608 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.731660 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.731817 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.731854 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3763,"thread_start_us":65,"threads_started":1}
I20260924 21:19:22.731920 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.725226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45835 (local address 127.0.0.1:57352)
0924 21:19:22.728902 (+ 3676us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.728930 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.728983 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.729215 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.729218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.729221 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:22.729278 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.729280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.730182 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.730183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.730964 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.730966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.731607 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.731608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.731687 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.731699 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.731800 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.731854 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3636,"thread_start_us":90,"threads_started":1}
I20260924 21:19:22.732468 2863 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8865fa3ba4476cb309fcc4c8241f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:22 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:57352
I20260924 21:19:22.732954 2938 tablet_replica.cc:330] stopping tablet replica
I20260924 21:19:22.733085 2938 raft_consensus.cc:2229] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:19:22.733152 2938 raft_consensus.cc:2258] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:22.733439 2938 ts_tablet_manager.cc:1905] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:19:22.735532 2938 ts_tablet_manager.cc:1918] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:19:22.735616 2938 log.cc:1196] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/7e8865fa3ba4476cb309fcc4c8241f16
I20260924 21:19:22.735854 2938 ts_tablet_manager.cc:1939] T 7e8865fa3ba4476cb309fcc4c8241f16 P bd62766ef9474d1d9c1d51729d9aea66: Deleting consensus metadata
I20260924 21:19:22.736351 2726 catalog_manager.cc:4841] TS bd62766ef9474d1d9c1d51729d9aea66 (127.20.115.193:45835): tablet 7e8865fa3ba4476cb309fcc4c8241f16 (table test-table [id=d06d43db62aa4d6bba6b40be9178825d]) successfully deleted
W20260924 21:19:22.804014 2933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:22.804240 2933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:22.804265 2933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:22.805534 2933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:22.805580 2933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:22.805611 2933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:19:22.807066 2933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:39767
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=38963
--tserver_master_addrs=127.20.115.254:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:22.807317 2933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:22.807551 2933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:22.812412 2944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:22.812419 2946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:22.812419 2943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:22.812870 2933 server_base.cc:1034] running on GCE node
I20260924 21:19:22.813063 2933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:22.813247 2933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:22.814383 2933 hybrid_clock.cc:648] HybridClock initialized: now 1790284762814366 us; error 31 us; skew 500 ppm
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:22.816993 2933 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:19:22.817414 2933 webserver.cc:460] Webserver started at http://127.20.115.194:38963/ using document root <none> and password file <none>
I20260924 21:19:22.817639 2933 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:22.817714 2933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:22.819196 2933 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:19:22.819926 2953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.820122 2933 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:19:22.820192 2933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "75013a6d236b4f36a1d8bf56aacf76e8"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "5dea4c45ccc61d7ca7dde152bc85a306"
server_key_iv: "6dc0fc3e6c4e231045fa0ecefd507b5a"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.820535 2933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:22.834884 2933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:22.837188 2933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:22.837332 2933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:22.837575 2933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:22.838008 2960 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:19:22.839423 2933 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:19:22.839483 2933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:19:22.839526 2933 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:19:22.840142 2933 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:19:22.840197 2933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 21:19:22.840261 2960 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Bootstrap starting.
I20260924 21:19:22.841753 2960 log.cc:824] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:22.844710 2960 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:19:22.844995 2960 tablet_bootstrap.cc:492] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Bootstrap complete.
I20260924 21:19:22.845136 2960 ts_tablet_manager.cc:1397] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260924 21:19:22.846324 2960 raft_consensus.cc:348] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.846536 2960 raft_consensus.cc:729] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75013a6d236b4f36a1d8bf56aacf76e8, State: Initialized, Role: FOLLOWER
I20260924 21:19:22.846690 2960 consensus_queue.cc:260] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } }
I20260924 21:19:22.846994 2960 ts_tablet_manager.cc:1428] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:19:22.847268 2933 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:39767
I20260924 21:19:22.847318 3069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:39767 every 8 connection(s)
I20260924 21:19:22.847615 2933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:19:22.849190 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 2933
I20260924 21:19:22.849472 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:38527
--local_ip_for_outbound_sockets=127.20.115.195
--tserver_master_addrs=127.20.115.254:33809
--webserver_port=34781
--webserver_interface=127.20.115.195
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:22.857661 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.848193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:52557)
0924 21:19:22.848347 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.848378 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.848504 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.848669 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.848672 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.848994 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:22.849183 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.849188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.850142 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.850143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.851191 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.851194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.851563 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.851564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.851723 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.852295 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:22.852303 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:22.854363 (+ 2060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:22.856270 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.856275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.856277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.856490 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.856706 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.856706 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.856707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.856862 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.857168 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:22.857173 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:22.857299 (+ 126us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.857377 (+ 78us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.857498 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260924 21:19:22.857676 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.848240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:52557 (local address 127.20.115.254:33809)
0924 21:19:22.848317 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.848320 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.848461 (+ 141us) server_negotiation.cc:407] Connection header received
0924 21:19:22.848564 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.848565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.848581 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.848631 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:22.849278 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.850029 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.851281 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.851479 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.851534 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.854462 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:22.854463 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:22.854464 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:22.854472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:22.856177 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.856564 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.856565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.856566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.856632 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.856929 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.856930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.856931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.857095 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:22.857176 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.857444 (+ 268us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.857545 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260924 21:19:22.858283 3070 heartbeater.cc:347] Connected to a master server at 127.20.115.254:33809
I20260924 21:19:22.858407 3070 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:22.858618 3070 heartbeater.cc:511] Master 127.20.115.254:33809 requested a full tablet report, sending...
I20260924 21:19:22.859186 2741 ts_manager.cc:194] Registered new tserver with Master: 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767)
I20260924 21:19:22.859843 2741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:52557
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.194@KRBTEST.COM
I20260924 21:19:22.867108 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.859518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:39767 (local address 127.0.0.1:37780)
0924 21:19:22.859571 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.859582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.859622 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.859893 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.859894 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.860244 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:22.860279 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.860281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.861203 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.861204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.861713 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.861715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.862199 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.862200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.862261 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.862777 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:22.862782 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:22.864158 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:22.865970 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.865976 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.865978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.866174 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.866407 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.866408 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.866409 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.866468 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.866760 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:22.866767 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:22.866817 (+ 50us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.866905 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.867012 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:19:22.867132 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.859571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37780 (local address 127.20.115.194:39767)
0924 21:19:22.859742 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.859746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.859755 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:19:22.859792 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.859794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.859805 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.859878 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:22.860361 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.861122 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.861798 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.862118 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.862200 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.864276 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:22.864282 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:22.864282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:22.864289 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:22.865865 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.866273 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.866276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.866277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.866336 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.866541 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.866542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.866542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.866682 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:22.866769 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.866939 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.867026 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260924 21:19:22.867566 3003 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e8865fa3ba4476cb309fcc4c8241f16 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:22 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:37780
I20260924 21:19:22.868124 3075 tablet_replica.cc:330] stopping tablet replica
I20260924 21:19:22.868233 3075 raft_consensus.cc:2229] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:19:22.868300 3075 raft_consensus.cc:2258] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:22.868605 3075 ts_tablet_manager.cc:1905] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:19:22.871061 3075 ts_tablet_manager.cc:1918] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:19:22.871136 3075 log.cc:1196] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/7e8865fa3ba4476cb309fcc4c8241f16
I20260924 21:19:22.871363 3075 ts_tablet_manager.cc:1939] T 7e8865fa3ba4476cb309fcc4c8241f16 P 75013a6d236b4f36a1d8bf56aacf76e8: Deleting consensus metadata
I20260924 21:19:22.871848 2728 catalog_manager.cc:4841] TS 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767): tablet 7e8865fa3ba4476cb309fcc4c8241f16 (table test-table [id=d06d43db62aa4d6bba6b40be9178825d]) successfully deleted
W20260924 21:19:22.934978 3073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:22.935166 3073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:22.935189 3073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:22.936254 3073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:22.936288 3073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:22.936313 3073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:19:22.937773 3073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:38527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=34781
--tserver_master_addrs=127.20.115.254:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:22.937943 3073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:22.938107 3073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:22.942891 3083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:22.942899 3081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:22.942934 3080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:22.943159 3073 server_base.cc:1034] running on GCE node
I20260924 21:19:22.943360 3073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:22.943615 3073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:22.944806 3073 hybrid_clock.cc:648] HybridClock initialized: now 1790284762944778 us; error 40 us; skew 500 ppm
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:22.947156 3073 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:19:22.947566 3073 webserver.cc:460] Webserver started at http://127.20.115.195:34781/ using document root <none> and password file <none>
I20260924 21:19:22.947790 3073 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:22.947871 3073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:22.949249 3073 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:19:22.949898 3090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.950145 3073 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:19:22.950241 3073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "d40f314fcd3f461ab1039758e46aa2d1"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "eff01ee92e6ba67275240b316c1121c1"
server_key_iv: "75f1c9c1d57135524acfdef70d2c3d90"
server_key_version: "encryptionkey@0"
I20260924 21:19:22.950587 3073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:22.959012 3073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:22.960718 3073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:22.960883 3073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:22.961097 3073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:22.961371 3073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:22.961423 3073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.961462 3073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:22.961496 3073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.966913 3073 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:38527
I20260924 21:19:22.966985 3203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:38527 every 8 connection(s)
I20260924 21:19:22.967296 3073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:19:22 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284762, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:22.974509 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 3073
I20260924 21:19:22.976307 2932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.968022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:34165 (local address 127.20.115.254:33809)
0924 21:19:22.968087 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.968089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.968199 (+ 110us) server_negotiation.cc:407] Connection header received
0924 21:19:22.968662 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.968663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.968672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.968731 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:22.969654 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.970130 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.971078 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.971268 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.971325 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.973357 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:22.973359 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:22.973359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:22.973366 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:22.974949 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.975388 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.975390 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.975390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.975449 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.975678 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.975679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.975679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.975813 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:22.975876 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.976185 (+ 309us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.976246 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:19:22.976506 3206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.967918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:34165)
0924 21:19:22.968066 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.968106 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.968583 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.968849 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.968860 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.969386 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:22.969573 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.969577 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.970205 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.970206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.970973 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.970975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.971340 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.971341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.971434 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.971864 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:22.971872 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:22.973268 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:22.975024 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.975032 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.975033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.975318 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.975519 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:22.975519 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:22.975520 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:22.975615 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:22.975879 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:22.975884 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:22.976024 (+ 140us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.976161 (+ 137us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.976308 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":89,"threads_started":1}
I20260924 21:19:22.976939 3204 heartbeater.cc:347] Connected to a master server at 127.20.115.254:33809
I20260924 21:19:22.977026 3204 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:22.977211 3204 heartbeater.cc:511] Master 127.20.115.254:33809 requested a full tablet report, sending...
I20260924 21:19:22.977587 2741 ts_manager.cc:194] Registered new tserver with Master: d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527)
I20260924 21:19:22.977950 2741 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:34165
I20260924 21:19:22.981438 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.974907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.20.115.254:33809)
0924 21:19:22.975022 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.975024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.975033 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:19:22.975092 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.975093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.975106 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.975175 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:22.975865 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.976582 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.977574 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.977833 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.977901 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.979080 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:22.979081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:22.979081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:22.979088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:22.980257 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.980605 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.980606 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.980607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.980659 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:22.980905 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:22.980906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:22.980907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:22.981041 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:22.981222 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.981261 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.981341 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":48,"threads_started":1}
I20260924 21:19:22.982108 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:19:22.984869 3208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.982434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53028 (local address 127.20.115.254:33809)
0924 21:19:22.982496 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.982498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.982598 (+ 100us) server_negotiation.cc:407] Connection header received
0924 21:19:22.982672 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.982673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.982701 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.982793 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:19:22.982888 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.983366 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.984107 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.984310 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.984390 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.984565 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:19:22.984656 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:19:22.984742 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.984766 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.984786 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:19:22.985116 2741 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:53028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:19:22.986626 2741 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:19:22.988804 2863 tablet_service.cc:1467] Processing CreateTablet for tablet cd86b6b52f4840cb8981dabafcceddc5 (DEFAULT_TABLE table=test-table [id=bcc86ae57b854925a0521b0b64cec506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:22.989164 3003 tablet_service.cc:1467] Processing CreateTablet for tablet cd86b6b52f4840cb8981dabafcceddc5 (DEFAULT_TABLE table=test-table [id=bcc86ae57b854925a0521b0b64cec506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:22.989203 2863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd86b6b52f4840cb8981dabafcceddc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:22.989346 3003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd86b6b52f4840cb8981dabafcceddc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:22.990851 2821 tablet_bootstrap.cc:492] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Bootstrap starting.
I20260924 21:19:22.990934 2960 tablet_bootstrap.cc:492] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: Bootstrap starting.
I20260924 21:19:22.991180 2821 tablet_bootstrap.cc:654] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:22.991364 2960 tablet_bootstrap.cc:654] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:22.991796 2821 tablet_bootstrap.cc:492] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: No bootstrap required, opened a new log
I20260924 21:19:22.991842 2821 ts_tablet_manager.cc:1397] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260924 21:19:22.991936 2821 raft_consensus.cc:348] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:22.992023 2821 raft_consensus.cc:374] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:22.992067 2821 raft_consensus.cc:729] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62766ef9474d1d9c1d51729d9aea66, State: Initialized, Role: FOLLOWER
I20260924 21:19:22.992130 2821 consensus_queue.cc:260] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:22.992228 2821 ts_tablet_manager.cc:1428] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:22.992604 2960 tablet_bootstrap.cc:492] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: No bootstrap required, opened a new log
I20260924 21:19:22.992651 2960 ts_tablet_manager.cc:1397] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:19:22.992995 2960 raft_consensus.cc:348] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:22.993081 2960 raft_consensus.cc:374] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:22.993129 2960 raft_consensus.cc:729] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75013a6d236b4f36a1d8bf56aacf76e8, State: Initialized, Role: FOLLOWER
I20260924 21:19:22.993172 2960 consensus_queue.cc:260] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:22.993264 2960 ts_tablet_manager.cc:1428] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260924 21:19:22.993340 3070 heartbeater.cc:503] Master 127.20.115.254:33809 was elected leader, sending a full tablet report...
I20260924 21:19:22.993335 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.988398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:38527 (local address 127.0.0.1:53220)
0924 21:19:22.988432 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:22.988450 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:22.988485 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:22.989222 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:22.989223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:22.989227 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:22.989295 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.989297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.991019 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.991020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.992184 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:22.992187 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.993132 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.993133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:22.993199 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.993202 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:19:22.993230 (+ 28us) client_negotiation.cc:240] Negotiation successful
0924 21:19:22.993263 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260924 21:19:22.993669 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:22.988568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53220 (local address 127.20.115.195:38527)
0924 21:19:22.988740 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:22.988744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:22.988755 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:19:22.988820 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:22.988822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:22.988838 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:22.988973 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:22.989408 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.990854 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.992289 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:22.993047 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:22.993449 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:22.993513 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:22.993536 (+ 23us) server_negotiation.cc:299] Negotiation successful
0924 21:19:22.993572 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260924 21:19:22.994194 3138 tablet_service.cc:1467] Processing CreateTablet for tablet cd86b6b52f4840cb8981dabafcceddc5 (DEFAULT_TABLE table=test-table [id=bcc86ae57b854925a0521b0b64cec506]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:22.994431 3138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cd86b6b52f4840cb8981dabafcceddc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:22.997124 3213 tablet_bootstrap.cc:492] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: Bootstrap starting.
I20260924 21:19:22.997682 3213 tablet_bootstrap.cc:654] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:22.998210 3213 log.cc:824] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:22.998867 3213 tablet_bootstrap.cc:492] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: No bootstrap required, opened a new log
I20260924 21:19:22.998947 3213 ts_tablet_manager.cc:1397] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:19:23.000237 3213 raft_consensus.cc:348] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:23.000385 3213 raft_consensus.cc:374] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:23.000432 3213 raft_consensus.cc:729] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d40f314fcd3f461ab1039758e46aa2d1, State: Initialized, Role: FOLLOWER
I20260924 21:19:23.000568 3213 consensus_queue.cc:260] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:23.000895 3213 ts_tablet_manager.cc:1428] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:19:23.000941 3204 heartbeater.cc:503] Master 127.20.115.254:33809 was elected leader, sending a full tablet report...
I20260924 21:19:23.025949 3064 raft_consensus.cc:482] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:23.026089 3064 raft_consensus.cc:504] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:23.026470 3064 leader_election.cc:288] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527), bd62766ef9474d1d9c1d51729d9aea66 (127.20.115.193:45835)
I20260924 21:19:23.030408 2937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.026905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:48755 (local address 127.20.115.193:45835)
0924 21:19:23.026949 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:23.026951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:23.027012 (+ 61us) server_negotiation.cc:407] Connection header received
0924 21:19:23.027073 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:23.027074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:23.027085 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:23.027153 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:23.027277 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.027962 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.029434 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.030019 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.030098 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:23.030107 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:23.030304 (+ 197us) server_negotiation.cc:299] Negotiation successful
0924 21:19:23.030349 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:19:23.030408 3210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.026745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:41967 (local address 127.20.115.195:38527)
0924 21:19:23.026814 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:23.026816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:23.026826 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:19:23.026882 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:23.026883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:23.026893 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:23.026983 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:23.027154 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.027913 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.029434 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.030013 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.030100 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:23.030107 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:23.030299 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 21:19:23.030340 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:19:23.030428 3072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.026662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:38527 (local address 127.20.115.194:41967)
0924 21:19:23.026720 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.026731 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:23.026801 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:23.026994 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:23.026995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:23.026996 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:23.027077 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:23.027079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.027992 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.027992 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:23.029311 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:23.029316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.030118 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.030119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:23.030196 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:23.030218 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 21:19:23.030280 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 21:19:23.030333 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":25}
I20260924 21:19:23.030439 3216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.026782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45835 (local address 127.20.115.194:48755)
0924 21:19:23.026946 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.026957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:23.027010 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:23.027161 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:23.027162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:23.027163 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:23.027217 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:23.027218 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.028038 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.028039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:23.029316 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:23.029318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.030136 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.030137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:23.030196 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:23.030218 (+ 22us) client_negotiation.cc:769] Sending connection context
0924 21:19:23.030280 (+ 62us) client_negotiation.cc:240] Negotiation successful
0924 21:19:23.030333 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":16,"thread_start_us":111,"threads_started":1}
I20260924 21:19:23.030815 3158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd86b6b52f4840cb8981dabafcceddc5" candidate_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40f314fcd3f461ab1039758e46aa2d1" is_pre_election: true
I20260924 21:19:23.030978 3158 raft_consensus.cc:2454] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75013a6d236b4f36a1d8bf56aacf76e8 in term 0.
I20260924 21:19:23.031210 2955 leader_election.cc:302] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75013a6d236b4f36a1d8bf56aacf76e8, d40f314fcd3f461ab1039758e46aa2d1; no voters:
I20260924 21:19:23.031172 2883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd86b6b52f4840cb8981dabafcceddc5" candidate_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62766ef9474d1d9c1d51729d9aea66" is_pre_election: true
I20260924 21:19:23.031322 2883 raft_consensus.cc:2454] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75013a6d236b4f36a1d8bf56aacf76e8 in term 0.
I20260924 21:19:23.031456 3064 raft_consensus.cc:2781] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:19:23.031520 3064 raft_consensus.cc:482] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:19:23.031543 3064 raft_consensus.cc:3037] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:23.032239 3064 raft_consensus.cc:504] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:23.032337 3064 leader_election.cc:288] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [CANDIDATE]: Term 1 election: Requested vote from peers d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527), bd62766ef9474d1d9c1d51729d9aea66 (127.20.115.193:45835)
I20260924 21:19:23.032519 3158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd86b6b52f4840cb8981dabafcceddc5" candidate_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d40f314fcd3f461ab1039758e46aa2d1"
I20260924 21:19:23.032533 2883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cd86b6b52f4840cb8981dabafcceddc5" candidate_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd62766ef9474d1d9c1d51729d9aea66"
I20260924 21:19:23.032624 3158 raft_consensus.cc:3037] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:23.032624 2883 raft_consensus.cc:3037] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:23.033303 2883 raft_consensus.cc:2454] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75013a6d236b4f36a1d8bf56aacf76e8 in term 1.
I20260924 21:19:23.033448 3158 raft_consensus.cc:2454] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75013a6d236b4f36a1d8bf56aacf76e8 in term 1.
I20260924 21:19:23.033488 2955 leader_election.cc:302] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75013a6d236b4f36a1d8bf56aacf76e8, bd62766ef9474d1d9c1d51729d9aea66; no voters:
I20260924 21:19:23.033572 3064 raft_consensus.cc:2781] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:19:23.033723 3064 raft_consensus.cc:686] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 LEADER]: Becoming Leader. State: Replica: 75013a6d236b4f36a1d8bf56aacf76e8, State: Running, Role: LEADER
I20260924 21:19:23.033818 3064 consensus_queue.cc:237] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:23.034520 2741 catalog_manager.cc:5495] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194). New cstate: current_term: 1 leader_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } health_report { overall_health: UNKNOWN } } }
I20260924 21:19:23.039772 3074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.037762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37796 (local address 127.20.115.194:39767)
0924 21:19:23.037813 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:23.037815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:23.037822 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:19:23.037869 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:23.037870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:23.037883 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:23.037939 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:19:23.038036 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.038481 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.039134 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:23.039317 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:23.039364 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:23.039493 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:19:23.039574 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:19:23.039626 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:23.039663 (+ 37us) server_negotiation.cc:299] Negotiation successful
0924 21:19:23.039679 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:19:23.041074 2883 raft_consensus.cc:1261] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Refusing update from remote peer 75013a6d236b4f36a1d8bf56aacf76e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:23.041188 3158 raft_consensus.cc:1261] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [term 1 FOLLOWER]: Refusing update from remote peer 75013a6d236b4f36a1d8bf56aacf76e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:23.041417 3064 consensus_queue.cc:1035] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:23.041549 3217 consensus_queue.cc:1035] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:23.043354 3220 mvcc.cc:205] Tried to move back new op lower bound from 7333006389414297600 to 7333006389387063296. Current Snapshot: MvccSnapshot[applied={T|T < 7333006389414297600}]
I20260924 21:19:23.043476 3221 mvcc.cc:205] Tried to move back new op lower bound from 7333006389414297600 to 7333006389387063296. Current Snapshot: MvccSnapshot[applied={T|T < 7333006389414297600}]
I20260924 21:19:23.044495 3222 mvcc.cc:205] Tried to move back new op lower bound from 7333006389414297600 to 7333006389387063296. Current Snapshot: MvccSnapshot[applied={T|T < 7333006389414297600}]
I20260924 21:19:23.045697 2741 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:53028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:19:23.045785 2741 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:53028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:19:23.046499 2741 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Sending DeleteTablet for 3 replicas of tablet cd86b6b52f4840cb8981dabafcceddc5
I20260924 21:19:23.046792 3138 tablet_service.cc:1514] Processing DeleteTablet for tablet cd86b6b52f4840cb8981dabafcceddc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:23 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:53220
I20260924 21:19:23.046823 3003 tablet_service.cc:1514] Processing DeleteTablet for tablet cd86b6b52f4840cb8981dabafcceddc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:23 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:37780
I20260924 21:19:23.046821 2863 tablet_service.cc:1514] Processing DeleteTablet for tablet cd86b6b52f4840cb8981dabafcceddc5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:23 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:57352
I20260924 21:19:23.046897 3075 tablet_replica.cc:330] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: stopping tablet replica
I20260924 21:19:23.046905 2938 tablet_replica.cc:330] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: stopping tablet replica
I20260924 21:19:23.046952 2938 raft_consensus.cc:2229] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:19:23.046952 3075 raft_consensus.cc:2229] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:19:23.046998 2938 raft_consensus.cc:2258] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:23.047004 3230 tablet_replica.cc:330] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: stopping tablet replica
I20260924 21:19:23.047030 3075 raft_consensus.cc:2258] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:23.047091 3230 raft_consensus.cc:2229] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:19:23.047179 3230 raft_consensus.cc:2258] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:23.047325 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 2779
I20260924 21:19:23.047375 3075 ts_tablet_manager.cc:1905] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:19:23.047551 3230 ts_tablet_manager.cc:1905] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:19:23.049144 3075 ts_tablet_manager.cc:1918] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:19:23.049156 3230 ts_tablet_manager.cc:1918] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:19:23.049197 3075 log.cc:1196] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/cd86b6b52f4840cb8981dabafcceddc5
I20260924 21:19:23.049217 3230 log.cc:1196] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/cd86b6b52f4840cb8981dabafcceddc5
I20260924 21:19:23.049425 3075 ts_tablet_manager.cc:1939] T cd86b6b52f4840cb8981dabafcceddc5 P 75013a6d236b4f36a1d8bf56aacf76e8: Deleting consensus metadata
I20260924 21:19:23.049492 3230 ts_tablet_manager.cc:1939] T cd86b6b52f4840cb8981dabafcceddc5 P d40f314fcd3f461ab1039758e46aa2d1: Deleting consensus metadata
I20260924 21:19:23.049981 2728 catalog_manager.cc:4841] TS 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767): tablet cd86b6b52f4840cb8981dabafcceddc5 (table test-table [id=bcc86ae57b854925a0521b0b64cec506]) successfully deleted
I20260924 21:19:23.050153 2726 catalog_manager.cc:4841] TS d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527): tablet cd86b6b52f4840cb8981dabafcceddc5 (table test-table [id=bcc86ae57b854925a0521b0b64cec506]) successfully deleted
W20260924 21:19:23.053944 2726 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:45835 (error 108)
I20260924 21:19:23.054082 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 2933
I20260924 21:19:23.054812 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.054292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45835 (local address 127.0.0.1:57354)
0924 21:19:23.054733 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.054773 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45835: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422}
W20260924 21:19:23.057240 2726 catalog_manager.cc:4584] TS bd62766ef9474d1d9c1d51729d9aea66 (127.20.115.193:45835): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cd86b6b52f4840cb8981dabafcceddc5: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45835: connect: Connection refused (error 111)
I20260924 21:19:23.058772 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 3073
I20260924 21:19:23.063766 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 2708
I20260924 21:19:23.068482 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33809
--webserver_interface=127.20.115.254
--webserver_port=36179
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:33809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:23.149256 3232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:23.149482 3232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:23.149518 3232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:23.150671 3232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:23.150739 3232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:23.150763 3232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:23.150792 3232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:23.150825 3232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:23.152272 3232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:33809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=36179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:23.152541 3232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:23.152765 3232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:23.157480 3240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:23.157493 3238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:23.157517 3237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:23.157568 3232 server_base.cc:1034] running on GCE node
I20260924 21:19:23.158010 3232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:23.158314 3232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:23.159485 3232 hybrid_clock.cc:648] HybridClock initialized: now 1790284763159469 us; error 32 us; skew 500 ppm
Sep 24 21:19:23 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284763, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:23.161975 3232 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:19:23.162411 3232 webserver.cc:460] Webserver started at http://127.20.115.254:36179/ using document root <none> and password file <none>
I20260924 21:19:23.162654 3232 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:23.162724 3232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:23.164049 3232 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:23.164669 3247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:23.164870 3232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:19:23.164971 3232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "a48862bb0f3749709affa358e2a028d3"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "f8c0d041aad2ce6569a2553d3ceef6a8"
server_key_iv: "76e03ca7429a5d7191d25cfaede23552"
server_key_version: "encryptionkey@0"
I20260924 21:19:23.165315 3232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:23.174408 3232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:23.176757 3232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:23.176995 3232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:23.180665 3232 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:33809
I20260924 21:19:23.180697 3299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:33809 every 8 connection(s)
I20260924 21:19:23.181097 3232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:19:23.182724 3300 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:19:23.182716 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 3232
I20260924 21:19:23.183015 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:45835
--local_ip_for_outbound_sockets=127.20.115.193
--tserver_master_addrs=127.20.115.254:33809
--webserver_port=45009
--webserver_interface=127.20.115.193
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:19:23.183725 3300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Bootstrap starting.
I20260924 21:19:23.192073 3300 log.cc:824] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:23.194290 3300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:19:23.194578 3300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3: Bootstrap complete.
I20260924 21:19:23.196885 3300 raft_consensus.cc:348] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:23.197158 3300 raft_consensus.cc:729] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48862bb0f3749709affa358e2a028d3, State: Initialized, Role: FOLLOWER
I20260924 21:19:23.197335 3300 consensus_queue.cc:260] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:23.197392 3300 raft_consensus.cc:388] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:19:23.197453 3300 raft_consensus.cc:482] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:19:23.197499 3300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 2 FOLLOWER]: Advancing to term 3
I20260924 21:19:23.198515 3300 raft_consensus.cc:504] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:23.198625 3300 leader_election.cc:302] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a48862bb0f3749709affa358e2a028d3; no voters:
I20260924 21:19:23.198815 3300 leader_election.cc:288] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 21:19:23.198848 3304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 21:19:23.198971 3304 raft_consensus.cc:686] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [term 3 LEADER]: Becoming Leader. State: Replica: a48862bb0f3749709affa358e2a028d3, State: Running, Role: LEADER
I20260924 21:19:23.199052 3304 consensus_queue.cc:237] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } }
I20260924 21:19:23.199098 3300 sys_catalog.cc:564] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:19:23.199286 3306 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a48862bb0f3749709affa358e2a028d3. Latest consensus state: current_term: 3 leader_uuid: "a48862bb0f3749709affa358e2a028d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } } }
I20260924 21:19:23.199535 3306 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:23.199474 3305 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a48862bb0f3749709affa358e2a028d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a48862bb0f3749709affa358e2a028d3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33809 } } }
I20260924 21:19:23.199781 3305 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48862bb0f3749709affa358e2a028d3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:23.200001 3313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:19:23.200816 3313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:19:23.201007 3313 catalog_manager.cc:1259] Loaded cluster ID: 11906873e8524c428caf319a1cd9e79c
I20260924 21:19:23.201145 3313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260924 21:19:23.201453 3313 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f057fb459ad google::LogMessage::Fail() at ??:0
@ 0x7f057fb4888a google::LogMessage::SendToLog() at ??:0
@ 0x7f057fb454de google::LogMessage::Flush() at ??:0
@ 0x7f057fb46a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0587a6a0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f0587a738b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0581a130d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0581a08954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f058286c6db start_thread at ??:0
@ 0x7f057e64361f clone at ??:0
W20260924 21:19:23.274880 3302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:23.275133 3302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:23.275174 3302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:23.276335 3302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:23.276396 3302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:23.276440 3302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:19:23.278023 3302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:45835
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=45009
--tserver_master_addrs=127.20.115.254:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:23.278290 3302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:23.278512 3302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:23.283370 3336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:23.283376 3337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:23.283425 3339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:23.283728 3302 server_base.cc:1034] running on GCE node
I20260924 21:19:23.283926 3302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:23.284183 3302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:23.285331 3302 hybrid_clock.cc:648] HybridClock initialized: now 1790284763285313 us; error 34 us; skew 500 ppm
Sep 24 21:19:23 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284763, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:23.292156 3302 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:19:23.292703 3302 webserver.cc:460] Webserver started at http://127.20.115.193:45009/ using document root <none> and password file <none>
I20260924 21:19:23.292994 3302 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:23.293044 3302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:23.294376 3302 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:23.295090 3346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:23.295255 3302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:23.295365 3302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "bd62766ef9474d1d9c1d51729d9aea66"
format_stamp: "Formatted at 2026-09-24 21:19:21 on dist-test-slave-cxpd"
server_key: "3938e2974adf449fe028a203e0b0ac27"
server_key_iv: "ea968c02126d7e7ea658ba8d027c3aec"
server_key_version: "encryptionkey@0"
I20260924 21:19:23.295713 3302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:23.307080 3302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:23.309333 3302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:23.309479 3302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:23.309715 3302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:23.310179 3353 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:19:23.312386 3302 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:19:23.312450 3302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:19:23.312500 3302 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:19:23.313171 3302 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:19:23.313225 3302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 21:19:23.313297 3353 tablet_bootstrap.cc:492] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Bootstrap starting.
I20260924 21:19:23.320505 3302 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:45835
I20260924 21:19:23.320607 3460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:45835 every 8 connection(s)
I20260924 21:19:23.320952 3302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:19:23.323740 3353 log.cc:824] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:23.326211 3353 tablet_bootstrap.cc:492] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:19:23.326485 3353 tablet_bootstrap.cc:492] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Bootstrap complete.
I20260924 21:19:23.327440 3353 ts_tablet_manager.cc:1397] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.007s
I20260924 21:19:23.328429 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 3302
I20260924 21:19:23.328722 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:39767
--local_ip_for_outbound_sockets=127.20.115.194
--tserver_master_addrs=127.20.115.254:33809
--webserver_port=38963
--webserver_interface=127.20.115.194
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:19:23.328646 3353 raft_consensus.cc:348] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:23.329017 3353 raft_consensus.cc:729] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd62766ef9474d1d9c1d51729d9aea66, State: Initialized, Role: FOLLOWER
I20260924 21:19:23.329182 3353 consensus_queue.cc:260] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:23.329442 3353 ts_tablet_manager.cc:1428] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260924 21:19:23.409261 3466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:23.409466 3466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:23.409489 3466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:23.410668 3466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:23.410712 3466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:23.410738 3466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:19:23.412209 3466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:39767
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=38963
--tserver_master_addrs=127.20.115.254:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:23.412453 3466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:23.412698 3466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:23.417896 3471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:23.417958 3466 server_base.cc:1034] running on GCE node
W20260924 21:19:23.417943 3474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:23.418184 3472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:23.418393 3466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:23.418751 3466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:23.419934 3466 hybrid_clock.cc:648] HybridClock initialized: now 1790284763419918 us; error 33 us; skew 500 ppm
Sep 24 21:19:23 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284763, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:23.422839 3466 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:19:23.423251 3466 webserver.cc:460] Webserver started at http://127.20.115.194:38963/ using document root <none> and password file <none>
I20260924 21:19:23.423498 3466 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:23.423571 3466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:23.424935 3466 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:23.425621 3481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20260924 21:19:23.425709 3463 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:19:23.321651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:52883)
0924 21:19:23.321816 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.321858 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:23.322296 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:23.425514 (+103218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260924 21:19:23.425781 3466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
W20260924 21:19:23.425957 3461 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:33809 (0 consecutive failures): Network error: Failed to ping master at 127.20.115.254:33809: Client connection negotiation failed: client connection to 127.20.115.254:33809: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 21:19:23.425856 3466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "75013a6d236b4f36a1d8bf56aacf76e8"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "5dea4c45ccc61d7ca7dde152bc85a306"
server_key_iv: "6dc0fc3e6c4e231045fa0ecefd507b5a"
server_key_version: "encryptionkey@0"
I20260924 21:19:23.426299 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.426159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:35061)
0924 21:19:23.426219 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.426270 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260924 21:19:23.426299 3466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:23.426689 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.426581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:51233)
0924 21:19:23.426634 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.426663 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 21:19:23.426816 3461 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:19:23.436324 3466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:23.438256 3466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:23.438390 3466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:23.438625 3466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:23.438908 3466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:23.438941 3466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:23.438961 3466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:23.439007 3466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:23.445008 3466 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:39767
I20260924 21:19:23.445048 3594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:39767 every 8 connection(s)
I20260924 21:19:23.445358 3466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:19:23.446277 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.445930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:40655)
0924 21:19:23.446071 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.446147 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
W20260924 21:19:23.446645 3595 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:33809 (0 consecutive failures): Network error: Failed to ping master at 127.20.115.254:33809: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
I20260924 21:19:23.446910 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.446818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:46465)
0924 21:19:23.446865 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.446893 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 21:19:23.447196 3597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.447112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:55209)
0924 21:19:23.447153 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.447174 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 21:19:23.447289 3595 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:19:23.453821 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 3466
I20260924 21:19:23.454082 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:38527
--local_ip_for_outbound_sockets=127.20.115.195
--tserver_master_addrs=127.20.115.254:33809
--webserver_port=34781
--webserver_interface=127.20.115.195
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:32949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:23.532822 3598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:23.533041 3598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:23.533077 3598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:23.534191 3598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:23.534255 3598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:23.534300 3598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:19:23.535758 3598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:32949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:38527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=34781
--tserver_master_addrs=127.20.115.254:33809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:23.536012 3598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:23.536237 3598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:23.541114 3603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:23.541200 3604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:23.541113 3606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:23.541320 3598 server_base.cc:1034] running on GCE node
I20260924 21:19:23.541567 3598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:23.541806 3598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:23.542972 3598 hybrid_clock.cc:648] HybridClock initialized: now 1790284763542954 us; error 46 us; skew 500 ppm
Sep 24 21:19:23 dist-test-slave-cxpd krb5kdc[2164](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284763, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:23.545258 3598 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:19:23.545673 3598 webserver.cc:460] Webserver started at http://127.20.115.195:34781/ using document root <none> and password file <none>
I20260924 21:19:23.545893 3598 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:23.545958 3598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:23.547151 3598 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:23.547655 3613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:23.547789 3598 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:23.547890 3598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "d40f314fcd3f461ab1039758e46aa2d1"
format_stamp: "Formatted at 2026-09-24 21:19:22 on dist-test-slave-cxpd"
server_key: "eff01ee92e6ba67275240b316c1121c1"
server_key_iv: "75f1c9c1d57135524acfdef70d2c3d90"
server_key_version: "encryptionkey@0"
I20260924 21:19:23.548198 3598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:23.556200 3598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:23.557895 3598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:23.558056 3598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:23.558264 3598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:23.558549 3598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:23.558609 3598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:23.558647 3598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:23.558676 3598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:23.564498 3598 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:38527
I20260924 21:19:23.564530 3726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:38527 every 8 connection(s)
I20260924 21:19:23.564879 3598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:19:23.565874 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.565530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:37863)
0924 21:19:23.565678 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.565746 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
W20260924 21:19:23.566236 3727 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:33809 (0 consecutive failures): Network error: Failed to ping master at 127.20.115.254:33809: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
I20260924 21:19:23.566550 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.566459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:55899)
0924 21:19:23.566508 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.566533 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:19:23.566787 3729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:23.566740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:60777)
0924 21:19:23.566758 (+ 18us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:23.566774 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260924 21:19:23.566905 3727 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:19:23.568611 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 3598
I20260924 21:19:24.427404 3731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:24.427166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:43183)
0924 21:19:24.427322 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:24.427356 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":61,"threads_started":1}
I20260924 21:19:24.447829 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:24.447602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:38121)
0924 21:19:24.447746 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:24.447783 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260924 21:19:24.567430 3733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:24.567223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:42149)
0924 21:19:24.567343 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:24.567379 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260924 21:19:24.929315 3734 raft_consensus.cc:482] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:24.929445 3734 raft_consensus.cc:504] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:24.929771 3734 leader_election.cc:288] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527), 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767)
Sep 24 21:19:24 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284763, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.194@KRBTEST.COM
Sep 24 21:19:24 dist-test-slave-cxpd krb5kdc[2164](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284763, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.195@KRBTEST.COM
I20260924 21:19:24.941030 3737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:24.930141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:60121 (local address 127.20.115.195:38527)
0924 21:19:24.930288 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:24.930306 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:24.930341 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:19:24.930774 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:24.930776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:24.930816 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:24.931032 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:24.931726 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:24.932579 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:24.933655 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:24.934060 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:24.934198 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:24.936743 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:24.936747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:24.936747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:24.936759 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:24.938956 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:24.939621 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:24.939624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:24.939625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:24.939739 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:24.940076 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:24.940076 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:24.940077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:24.940389 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:24.940474 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:24.940759 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 21:19:24.940906 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":51,"threads_started":1}
I20260924 21:19:24.941016 3735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:24.929932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:39767 (local address 127.20.115.193:52119)
0924 21:19:24.930088 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:24.930099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:24.930171 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:24.931034 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:24.931037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:24.931417 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:24.931607 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:24.931609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:24.932690 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:24.932691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:24.933564 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:24.933567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:24.934173 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:24.934174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:24.934282 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:24.934945 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:24.934950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:24.936366 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:24.939047 (+ 2681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:24.939056 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:24.939058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:24.939514 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:24.939822 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:24.939823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:24.939824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:24.939992 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:24.940513 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:24.940519 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:24.940592 (+ 73us) client_negotiation.cc:769] Sending connection context
0924 21:19:24.940705 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:19:24.940918 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":52,"thread_start_us":49,"threads_started":1}
I20260924 21:19:24.941151 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:24.929955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:38527 (local address 127.20.115.193:60121)
0924 21:19:24.930092 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:24.930104 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:24.930161 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:24.931034 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:24.931037 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:24.931417 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:24.931601 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:24.931607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:24.932709 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:24.932710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:24.933563 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:24.933565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:24.934167 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:24.934168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:24.934288 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:24.934884 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:24.934892 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:24.936637 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:24.939047 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:24.939056 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:24.939058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:24.939515 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:24.939822 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:24.939823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:24.939824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:24.939992 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:24.940463 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:24.940469 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:24.940592 (+ 123us) client_negotiation.cc:769] Sending connection context
0924 21:19:24.940705 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:19:24.940999 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":170,"thread_start_us":77,"threads_started":1}
I20260924 21:19:24.941665 3738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:24.930197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:52119 (local address 127.20.115.194:39767)
0924 21:19:24.930342 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:24.930351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:24.930377 (+ 26us) server_negotiation.cc:407] Connection header received
0924 21:19:24.930803 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:24.930806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:24.930841 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:24.930995 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:24.931725 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:24.932580 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:24.933687 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:24.934065 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:24.934201 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:24.936742 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:24.936747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:24.936747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:24.936759 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:24.938962 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:24.939621 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:24.939624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:24.939625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:24.939739 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:24.940071 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:24.940072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:24.940072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:24.940389 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:24.940480 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:24.940766 (+ 286us) server_negotiation.cc:299] Negotiation successful
0924 21:19:24.940951 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
W20260924 21:19:24.941870 3348 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
W20260924 21:19:24.942330 3350 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
I20260924 21:19:24.942389 3350 leader_election.cc:302] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd62766ef9474d1d9c1d51729d9aea66; no voters: 75013a6d236b4f36a1d8bf56aacf76e8, d40f314fcd3f461ab1039758e46aa2d1
I20260924 21:19:24.942518 3734 raft_consensus.cc:2726] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:19:25.427979 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:25.427852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:59551)
0924 21:19:25.427907 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:25.427939 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:19:25.448359 3739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:25.448191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:40653)
0924 21:19:25.448290 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:25.448324 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"thread_start_us":47,"threads_started":1}
I20260924 21:19:25.568150 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:25.567911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:53917)
0924 21:19:25.568032 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:25.568100 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260924 21:19:26.428608 3741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:26.428413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:37517)
0924 21:19:26.428531 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:26.428562 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":62,"threads_started":1}
I20260924 21:19:26.449136 3742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:26.448909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:34213)
0924 21:19:26.449037 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:26.449085 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260924 21:19:26.568774 3743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:26.568559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:33135)
0924 21:19:26.568702 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:26.568737 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
I20260924 21:19:27.005761 3744 raft_consensus.cc:482] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:27.005875 3744 raft_consensus.cc:504] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:27.006191 3744 leader_election.cc:288] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527), 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767)
W20260924 21:19:27.006398 3348 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
W20260924 21:19:27.006522 3350 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
I20260924 21:19:27.006560 3350 leader_election.cc:302] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd62766ef9474d1d9c1d51729d9aea66; no voters: 75013a6d236b4f36a1d8bf56aacf76e8, d40f314fcd3f461ab1039758e46aa2d1
I20260924 21:19:27.006641 3744 raft_consensus.cc:2726] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:19:27.429512 3745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:27.429243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:36061)
0924 21:19:27.429406 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:27.429462 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":63,"threads_started":1}
I20260924 21:19:27.449771 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:27.449550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:34451)
0924 21:19:27.449685 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:27.449721 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":57,"threads_started":1}
I20260924 21:19:27.569434 3747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:27.569230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:45759)
0924 21:19:27.569362 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:27.569395 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":56,"threads_started":1}
I20260924 21:19:28.430239 3748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:28.430026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:51723)
0924 21:19:28.430144 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:28.430179 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260924 21:19:28.450476 3749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:28.450232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:52243)
0924 21:19:28.450395 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:28.450424 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":81,"threads_started":1}
I20260924 21:19:28.570123 3750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:28.569900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:49033)
0924 21:19:28.570030 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:28.570065 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260924 21:19:29.430887 3751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:29.430653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:58721)
0924 21:19:29.430768 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:29.430827 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260924 21:19:29.451068 3752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:29.450885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:45785)
0924 21:19:29.450997 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:29.451028 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":56,"threads_started":1}
I20260924 21:19:29.570789 3753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:29.570563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:39211)
0924 21:19:29.570711 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:29.570746 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":59,"threads_started":1}
I20260924 21:19:30.359617 3754 raft_consensus.cc:482] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:30.359715 3754 raft_consensus.cc:504] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:30.359900 3754 leader_election.cc:288] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527), 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767)
W20260924 21:19:30.360334 3350 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
W20260924 21:19:30.360400 3348 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
I20260924 21:19:30.360427 3350 leader_election.cc:302] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd62766ef9474d1d9c1d51729d9aea66; no voters: 75013a6d236b4f36a1d8bf56aacf76e8, d40f314fcd3f461ab1039758e46aa2d1
I20260924 21:19:30.360497 3754 raft_consensus.cc:2726] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:19:30.431546 3755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:30.431334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:59781)
0924 21:19:30.431459 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:30.431499 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":57,"threads_started":1}
I20260924 21:19:30.451750 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:30.451538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:52107)
0924 21:19:30.451676 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:30.451710 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260924 21:19:30.571485 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:30.571278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:39213)
0924 21:19:30.571405 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:30.571441 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260924 21:19:31.432453 3758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:31.432205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:60589)
0924 21:19:31.432365 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:31.432405 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":75,"threads_started":1}
I20260924 21:19:31.452373 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:31.452176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:42667)
0924 21:19:31.452300 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:31.452334 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260924 21:19:31.572253 3760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:31.572016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:49237)
0924 21:19:31.572168 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:31.572204 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":66,"threads_started":1}
I20260924 21:19:32.433218 3761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:32.432992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:53199)
0924 21:19:32.433129 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:32.433170 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260924 21:19:32.453058 3762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:32.452863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:60923)
0924 21:19:32.452988 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:32.453017 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":71,"threads_started":1}
I20260924 21:19:32.572996 3763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:32.572744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:48385)
0924 21:19:32.572911 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:32.572953 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260924 21:19:33.064406 3764 raft_consensus.cc:482] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:33.064538 3764 raft_consensus.cc:504] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:33.064750 3764 leader_election.cc:288] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527), 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767)
W20260924 21:19:33.065201 3350 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
W20260924 21:19:33.065280 3348 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
I20260924 21:19:33.065343 3348 leader_election.cc:302] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd62766ef9474d1d9c1d51729d9aea66; no voters: 75013a6d236b4f36a1d8bf56aacf76e8, d40f314fcd3f461ab1039758e46aa2d1
I20260924 21:19:33.065447 3764 raft_consensus.cc:2726] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:19:33.433977 3765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:33.433747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:39929)
0924 21:19:33.433889 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:33.433928 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260924 21:19:33.453720 3766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:33.453481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:33583)
0924 21:19:33.453630 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:33.453671 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":106,"threads_started":1}
I20260924 21:19:33.573807 3767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:33.573576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:53535)
0924 21:19:33.573716 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:33.573759 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":59,"threads_started":1}
I20260924 21:19:34.434751 3768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:34.434483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:54709)
0924 21:19:34.434660 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:34.434703 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":82,"threads_started":1}
I20260924 21:19:34.454377 3769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:34.454171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:37361)
0924 21:19:34.454295 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:34.454330 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":65,"threads_started":1}
I20260924 21:19:34.574523 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:34.574299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:59575)
0924 21:19:34.574438 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:34.574477 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
I20260924 21:19:35.435557 3771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:35.435331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:39693)
0924 21:19:35.435465 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:35.435507 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260924 21:19:35.455132 3772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:35.454885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:57127)
0924 21:19:35.455049 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:35.455087 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
I20260924 21:19:35.575269 3773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:35.575033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:33115)
0924 21:19:35.575185 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:35.575222 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
I20260924 21:19:36.436385 3774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:36.436168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:33485)
0924 21:19:36.436296 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:36.436339 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260924 21:19:36.455875 3775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:36.455652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:60343)
0924 21:19:36.455785 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:36.455830 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260924 21:19:36.576200 3776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:36.575874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:49929)
0924 21:19:36.576110 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:36.576153 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":120,"threads_started":1}
I20260924 21:19:36.755271 3777 raft_consensus.cc:482] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:36.755389 3777 raft_consensus.cc:504] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d40f314fcd3f461ab1039758e46aa2d1" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 38527 } } peers { permanent_uuid: "75013a6d236b4f36a1d8bf56aacf76e8" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39767 } } peers { permanent_uuid: "bd62766ef9474d1d9c1d51729d9aea66" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45835 } }
I20260924 21:19:36.755630 3777 leader_election.cc:288] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527), 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767)
W20260924 21:19:36.755965 3348 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d40f314fcd3f461ab1039758e46aa2d1 (127.20.115.195:38527): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
W20260924 21:19:36.756057 3350 leader_election.cc:341] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 75013a6d236b4f36a1d8bf56aacf76e8 (127.20.115.194:39767): Not found: Tablet not found: cd86b6b52f4840cb8981dabafcceddc5
I20260924 21:19:36.756109 3350 leader_election.cc:302] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd62766ef9474d1d9c1d51729d9aea66; no voters: 75013a6d236b4f36a1d8bf56aacf76e8, d40f314fcd3f461ab1039758e46aa2d1
I20260924 21:19:36.756184 3777 raft_consensus.cc:2726] T cd86b6b52f4840cb8981dabafcceddc5 P bd62766ef9474d1d9c1d51729d9aea66 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:19:37.437275 3778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:37.437057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:32997)
0924 21:19:37.437192 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:37.437228 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
I20260924 21:19:37.456570 3779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:37.456342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:54047)
0924 21:19:37.456488 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:37.456525 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260924 21:19:37.577088 3780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:37.576738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:35107)
0924 21:19:37.576979 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:37.577024 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":149,"threads_started":1}
I20260924 21:19:38.438148 3781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:38.437876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.193:35285)
0924 21:19:38.438059 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:38.438103 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":63,"threads_started":1}
I20260924 21:19:38.457386 3782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:38.457150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.194:59231)
0924 21:19:38.457298 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:38.457340 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260924 21:19:38.578024 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 3302
I20260924 21:19:38.578622 3783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:38.577817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:33809 (local address 127.20.115.195:43079)
0924 21:19:38.578050 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:38.578118 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:33809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":184,"threads_started":1}
I20260924 21:19:38.582763 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 3466
I20260924 21:19:38.587739 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 3598
2026-09-24T21:19:38Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18080 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:19:38 dist-test-slave-cxpd krb5kdc[3786](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:19:38 dist-test-slave-cxpd krb5kdc[3786](info): set up 2 sockets
Sep 24 21:19:38 dist-test-slave-cxpd krb5kdc[3786](info): commencing operation
krb5kdc: starting...
W20260924 21:19:39.617012 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:19:39 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:19:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:19:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:19:39.737004 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46259
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:46259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:39.815850 3799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:39.816076 3799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:39.816123 3799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:39.817271 3799 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 21:19:39.817338 3799 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 21:19:39.817360 3799 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 21:19:39.817379 3799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:19:39.817409 3799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:39.817431 3799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:39.817461 3799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:19:39.817488 3799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:19:39.818912 3799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:46259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:39.819182 3799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:39.819396 3799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:39.824127 3804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:39.824160 3807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:39.824165 3805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:39.824293 3799 server_base.cc:1034] running on GCE node
I20260924 21:19:39.824561 3799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:39.824862 3799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:39.826076 3799 hybrid_clock.cc:648] HybridClock initialized: now 1790284779826056 us; error 37 us; skew 500 ppm
Sep 24 21:19:39 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284779, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:39.828308 3799 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:19:39.828816 3799 webserver.cc:460] Webserver started at http://127.20.115.254:34215/ using document root <none> and password file <none>
I20260924 21:19:39.829067 3799 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:39.829137 3799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:39.829289 3799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:39.830296 3799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "80cba6bf686646d28a3e4fdbb6607fa3"
format_stamp: "Formatted at 2026-09-24 21:19:39 on dist-test-slave-cxpd"
server_key: "8ccd1ae2bf31de35f6ce6f87c49e3518"
server_key_iv: "875b81a36dd3e73f6ece38605cb5c614"
server_key_version: "encryptionkey@0"
I20260924 21:19:39.830662 3799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "80cba6bf686646d28a3e4fdbb6607fa3"
format_stamp: "Formatted at 2026-09-24 21:19:39 on dist-test-slave-cxpd"
server_key: "8ccd1ae2bf31de35f6ce6f87c49e3518"
server_key_iv: "875b81a36dd3e73f6ece38605cb5c614"
server_key_version: "encryptionkey@0"
I20260924 21:19:39.831947 3799 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:19:39.832737 3814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:39.832954 3799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:39.833058 3799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "80cba6bf686646d28a3e4fdbb6607fa3"
format_stamp: "Formatted at 2026-09-24 21:19:39 on dist-test-slave-cxpd"
server_key: "8ccd1ae2bf31de35f6ce6f87c49e3518"
server_key_iv: "875b81a36dd3e73f6ece38605cb5c614"
server_key_version: "encryptionkey@0"
I20260924 21:19:39.833148 3799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:39.848090 3799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:39.849951 3799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:39.850123 3799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:39.854313 3799 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:46259
I20260924 21:19:39.854347 3866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:46259 every 8 connection(s)
I20260924 21:19:39.854699 3799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:19:39.855538 3867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:39.857774 3867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Bootstrap starting.
I20260924 21:19:39.858309 3867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:39.858536 3867 log.cc:824] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:39.859140 3867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: No bootstrap required, opened a new log
I20260924 21:19:39.860589 3867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:19:39.860720 3867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:39.860743 3867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80cba6bf686646d28a3e4fdbb6607fa3, State: Initialized, Role: FOLLOWER
I20260924 21:19:39.860916 3867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:19:39.861002 3867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:19:39.861038 3867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:19:39.861100 3867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:39.861564 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 3799
I20260924 21:19:39.861701 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:19:39.861752 3867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:19:39.861864 3867 leader_election.cc:302] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80cba6bf686646d28a3e4fdbb6607fa3; no voters:
I20260924 21:19:39.861908 20943 external_mini_cluster.cc:1428] Setting key a6e730c8951bf41fdce445adeeb41f32
I20260924 21:19:39.862030 3867 leader_election.cc:288] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:19:39.862087 3870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:19:39.862251 3870 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 1 LEADER]: Becoming Leader. State: Replica: 80cba6bf686646d28a3e4fdbb6607fa3, State: Running, Role: LEADER
I20260924 21:19:39.862280 3867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:19:39.862340 3870 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:19:39.862689 3870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80cba6bf686646d28a3e4fdbb6607fa3. Latest consensus state: current_term: 1 leader_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } } }
I20260924 21:19:39.862835 3870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:39.862895 3871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } } }
I20260924 21:19:39.863024 3871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:19:39.863164 3881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:19:39.863698 3881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:19:39.865295 3881 catalog_manager.cc:1347] Generated new cluster ID: 4804670773f74f50a65f73e792a44b67
I20260924 21:19:39.865345 3881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:19:39 dist-test-slave-cxpd krb5kdc[3786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:39.873999 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:39.862754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49318 (local address 127.20.115.254:46259)
0924 21:19:39.863251 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:39.863261 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:39.863297 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:19:39.863428 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:39.863436 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:39.863501 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:39.863667 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:39.864391 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:39.865772 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:39.866809 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:39.867428 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:39.867541 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:39.870303 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:39.870307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:39.870307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:39.870318 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:39.872284 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:39.872704 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:39.872707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:39.872707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:39.872842 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:39.873064 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:39.873065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:39.873065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:39.873363 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:39.873498 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:39.873643 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 21:19:39.873771 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":66,"threads_started":1}
I20260924 21:19:39.896850 3881 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:19:39.897418 3881 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:19:39.907349 3881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Generated new TSK 0
I20260924 21:19:39.907549 3881 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:19:39.940256 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46259
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:40.021003 3893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:40.021204 3893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:40.021229 3893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:40.022393 3893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:40.022433 3893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:40.022493 3893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:19:40.023926 3893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:40.024184 3893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:40.024441 3893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:40.029600 3901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:40.029619 3899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:40.029737 3893 server_base.cc:1034] running on GCE node
W20260924 21:19:40.029798 3898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:40.030002 3893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:40.030222 3893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:40.031404 3893 hybrid_clock.cc:648] HybridClock initialized: now 1790284780031385 us; error 30 us; skew 500 ppm
Sep 24 21:19:40 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284780, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:40.033999 3893 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:19:40.034549 3893 webserver.cc:460] Webserver started at http://127.20.115.193:36697/ using document root <none> and password file <none>
I20260924 21:19:40.034798 3893 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:40.034870 3893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:40.035041 3893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:40.036157 3893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "117a8e037e7740818096e4852d0242dd"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "533a9481ecbe7b9ab3af901a36ec879d"
server_key_iv: "a86592a9788fae9c55936538eb50d48d"
server_key_version: "encryptionkey@0"
I20260924 21:19:40.036561 3893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "117a8e037e7740818096e4852d0242dd"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "533a9481ecbe7b9ab3af901a36ec879d"
server_key_iv: "a86592a9788fae9c55936538eb50d48d"
server_key_version: "encryptionkey@0"
I20260924 21:19:40.037935 3893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:19:40.038697 3908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:40.038879 3893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:19:40.038980 3893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "117a8e037e7740818096e4852d0242dd"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "533a9481ecbe7b9ab3af901a36ec879d"
server_key_iv: "a86592a9788fae9c55936538eb50d48d"
server_key_version: "encryptionkey@0"
I20260924 21:19:40.039088 3893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:40.059396 3893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:40.061259 3893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:40.061417 3893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:40.061638 3893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:40.061931 3893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:40.061988 3893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:40.062021 3893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:40.062057 3893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:40.067927 3893 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:38773
I20260924 21:19:40.067987 4021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:38773 every 8 connection(s)
I20260924 21:19:40.068271 3893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:19:40 dist-test-slave-cxpd krb5kdc[3786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284780, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:40.075201 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 3893
I20260924 21:19:40.075337 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:19:40.075573 20943 external_mini_cluster.cc:1428] Setting key 7910beabc69451b09985ba301cc6adb7
I20260924 21:19:40.078128 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.068966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:39759 (local address 127.20.115.254:46259)
0924 21:19:40.069043 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.069046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.069248 (+ 202us) server_negotiation.cc:407] Connection header received
0924 21:19:40.069707 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.069708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.069718 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.069764 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:40.070650 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.071161 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.072136 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.072348 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.072416 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.074753 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:40.074754 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:40.074755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:40.074761 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:40.076528 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:40.077057 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:40.077058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:40.077058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:40.077123 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:40.077403 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:40.077403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:40.077403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:40.077493 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:40.077539 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.077876 (+ 337us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.077981 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:19:40.078244 4024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.068938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:39759)
0924 21:19:40.069123 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:40.069157 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:40.069620 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:40.069822 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:40.069825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:40.070374 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:40.070550 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.070555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.071242 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.071243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.072023 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.072025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.072486 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.072486 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.072594 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.073194 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:40.073202 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:40.074630 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:40.076612 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:40.076618 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:40.076620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:40.076962 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:40.077197 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:40.077198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:40.077198 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:40.077343 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:40.077556 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:40.077561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:40.077724 (+ 163us) client_negotiation.cc:769] Sending connection context
0924 21:19:40.077835 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 21:19:40.078026 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":100,"threads_started":1}
I20260924 21:19:40.078719 4022 heartbeater.cc:347] Connected to a master server at 127.20.115.254:46259
I20260924 21:19:40.078817 4022 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:40.078992 4022 heartbeater.cc:511] Master 127.20.115.254:46259 requested a full tablet report, sending...
I20260924 21:19:40.079522 3831 ts_manager.cc:194] Registered new tserver with Master: 117a8e037e7740818096e4852d0242dd (127.20.115.193:38773)
I20260924 21:19:40.080135 3831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:39759
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:19:40.103762 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46259
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:40.183634 4027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:40.183883 4027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:40.183921 4027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:40.185091 4027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:40.185168 4027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:40.185214 4027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:19:40.186666 4027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:40.186939 4027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:40.187172 4027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:40.192276 4035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:40.192291 4027 server_base.cc:1034] running on GCE node
W20260924 21:19:40.192286 4032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:40.192273 4033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:40.192655 4027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:40.192921 4027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:40.194097 4027 hybrid_clock.cc:648] HybridClock initialized: now 1790284780194084 us; error 29 us; skew 500 ppm
Sep 24 21:19:40 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284780, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:40.196604 4027 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:19:40.197139 4027 webserver.cc:460] Webserver started at http://127.20.115.194:38607/ using document root <none> and password file <none>
I20260924 21:19:40.197386 4027 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:40.197459 4027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:40.197600 4027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:40.198662 4027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "ce32df00534b4e29a67dd0c2a11b0374"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "a638a2670610d8c7cffdd647a1acddda"
server_key_iv: "ecc7610177ca6f074035603e529633e0"
server_key_version: "encryptionkey@0"
I20260924 21:19:40.199025 4027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "ce32df00534b4e29a67dd0c2a11b0374"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "a638a2670610d8c7cffdd647a1acddda"
server_key_iv: "ecc7610177ca6f074035603e529633e0"
server_key_version: "encryptionkey@0"
I20260924 21:19:40.200366 4027 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:19:40.201306 4042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:40.201538 4027 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:19:40.201651 4027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "ce32df00534b4e29a67dd0c2a11b0374"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "a638a2670610d8c7cffdd647a1acddda"
server_key_iv: "ecc7610177ca6f074035603e529633e0"
server_key_version: "encryptionkey@0"
I20260924 21:19:40.201743 4027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:40.211993 4027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:40.213889 4027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:40.214092 4027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:40.214311 4027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:40.214617 4027 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:40.214675 4027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:40.214715 4027 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:40.214744 4027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:40.220634 4027 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:40509
I20260924 21:19:40.220676 4155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:40509 every 8 connection(s)
I20260924 21:19:40.221032 4027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:19:40 dist-test-slave-cxpd krb5kdc[3786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284780, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:40.228183 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 4027
I20260924 21:19:40.228319 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:19:40.228529 20943 external_mini_cluster.cc:1428] Setting key 8c12884d2c3af2ede5d7fc6d8b86f7f0
I20260924 21:19:40.230508 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.221703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:37235 (local address 127.20.115.254:46259)
0924 21:19:40.221790 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.221793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.221918 (+ 125us) server_negotiation.cc:407] Connection header received
0924 21:19:40.222491 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.222492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.222506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.222559 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:40.223475 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.223999 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.225100 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.225320 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.225417 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.227575 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:40.227577 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:40.227577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:40.227583 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:40.229050 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:40.229529 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:40.229530 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:40.229530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:40.229582 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:40.229871 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:40.229872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:40.229872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:40.229998 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:40.230052 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.230370 (+ 318us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.230443 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 21:19:40.230858 4158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.221635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:37235)
0924 21:19:40.221786 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:40.221822 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:40.222368 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:40.222628 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:40.222633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:40.223181 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:40.223363 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.223367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.224105 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.224106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.225003 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.225006 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.225428 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.225429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.225626 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.226159 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:40.226167 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:40.227482 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:40.229126 (+ 1644us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:40.229133 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:40.229135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:40.229447 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:40.229647 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:40.229648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:40.229648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:40.229805 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:40.230071 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:40.230076 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:40.230236 (+ 160us) client_negotiation.cc:769] Sending connection context
0924 21:19:40.230349 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:19:40.230522 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260924 21:19:40.231309 4156 heartbeater.cc:347] Connected to a master server at 127.20.115.254:46259
I20260924 21:19:40.231421 4156 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:40.231609 4156 heartbeater.cc:511] Master 127.20.115.254:46259 requested a full tablet report, sending...
I20260924 21:19:40.231971 3831 ts_manager.cc:194] Registered new tserver with Master: ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509)
I20260924 21:19:40.232369 3831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:37235
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:19:40.255527 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46259
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:19:40.335021 4161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:19:40.335208 4161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:19:40.335230 4161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:19:40.336416 4161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:19:40.336453 4161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:19:40.336521 4161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:19:40.337917 4161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:19:40.338088 4161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:19:40.338253 4161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:19:40.342969 4167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:19:40.342981 4166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:40.343087 4161 server_base.cc:1034] running on GCE node
W20260924 21:19:40.342962 4169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:19:40.343425 4161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:19:40.343619 4161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:19:40.344833 4161 hybrid_clock.cc:648] HybridClock initialized: now 1790284780344819 us; error 27 us; skew 500 ppm
Sep 24 21:19:40 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284780, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:19:40.347255 4161 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:19:40.347666 4161 webserver.cc:460] Webserver started at http://127.20.115.195:44731/ using document root <none> and password file <none>
I20260924 21:19:40.347885 4161 fs_manager.cc:359] Metadata directory not provided
I20260924 21:19:40.347956 4161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:19:40.348114 4161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:19:40.349227 4161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "c9c8c186b4f64b6686efa896ab96de40"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "085058bd3a211f4e13096e29747d7b2e"
server_key_iv: "00ec6ae40b26b468e64c815bb8aa77c6"
server_key_version: "encryptionkey@0"
I20260924 21:19:40.349604 4161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "c9c8c186b4f64b6686efa896ab96de40"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "085058bd3a211f4e13096e29747d7b2e"
server_key_iv: "00ec6ae40b26b468e64c815bb8aa77c6"
server_key_version: "encryptionkey@0"
I20260924 21:19:40.350986 4161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:19:40.351876 4176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:40.352113 4161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:19:40.352221 4161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "c9c8c186b4f64b6686efa896ab96de40"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "085058bd3a211f4e13096e29747d7b2e"
server_key_iv: "00ec6ae40b26b468e64c815bb8aa77c6"
server_key_version: "encryptionkey@0"
I20260924 21:19:40.352277 4161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:19:40.366073 4161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:19:40.368322 4161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:19:40.368430 4161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:19:40.368583 4161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:19:40.368943 4161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:19:40.368979 4161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:40.369000 4161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:19:40.369048 4161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:19:40.375084 4161 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:45245
I20260924 21:19:40.375173 4289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:45245 every 8 connection(s)
I20260924 21:19:40.375481 4161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:19:40.379877 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 4161
I20260924 21:19:40.379973 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:19:40.380148 20943 external_mini_cluster.cc:1428] Setting key 227a7297100b3564392344035e575104
Sep 24 21:19:40 dist-test-slave-cxpd krb5kdc[3786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284780, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:19:40.384449 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.376172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:35319 (local address 127.20.115.254:46259)
0924 21:19:40.376251 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.376253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.376423 (+ 170us) server_negotiation.cc:407] Connection header received
0924 21:19:40.376973 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.376974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.376983 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.377035 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:40.377888 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.378368 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.379288 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.379464 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.379520 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.381695 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:40.381696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:40.381697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:40.381704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:40.383086 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:40.383504 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:40.383505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:40.383505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:40.383562 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:40.383813 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:40.383814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:40.383814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:40.383929 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:40.384007 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.384285 (+ 278us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.384361 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:19:40.384617 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.376133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:35319)
0924 21:19:40.376309 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:40.376341 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:40.376874 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:40.377085 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:40.377088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:40.377630 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:19:40.377806 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.377810 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.378447 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.378447 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.379211 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.379213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.379540 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.379541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.379644 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.380241 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:19:40.380252 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:19:40.381590 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:19:40.383192 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:40.383198 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:40.383200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:40.383414 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:40.383630 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:19:40.383631 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:19:40.383631 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:19:40.383740 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:19:40.383999 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:19:40.384005 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:19:40.384140 (+ 135us) client_negotiation.cc:769] Sending connection context
0924 21:19:40.384250 (+ 110us) client_negotiation.cc:240] Negotiation successful
0924 21:19:40.384398 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260924 21:19:40.385085 4290 heartbeater.cc:347] Connected to a master server at 127.20.115.254:46259
I20260924 21:19:40.385175 4290 heartbeater.cc:464] Registering TS with master...
I20260924 21:19:40.385365 4290 heartbeater.cc:511] Master 127.20.115.254:46259 requested a full tablet report, sending...
I20260924 21:19:40.385722 3831 ts_manager.cc:194] Registered new tserver with Master: c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245)
I20260924 21:19:40.386087 3831 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:35319
I20260924 21:19:40.391738 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:19:40.398965 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.393055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49328 (local address 127.20.115.254:46259)
0924 21:19:40.393156 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.393158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.393269 (+ 111us) server_negotiation.cc:407] Connection header received
0924 21:19:40.393344 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.393345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.393354 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.393399 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:19:40.393943 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.394424 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.395191 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.395393 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.395470 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.396569 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:19:40.396570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:19:40.396570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:19:40.396576 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:19:40.397804 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:40.398175 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:40.398176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:40.398177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:40.398232 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:19:40.398435 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:19:40.398436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:19:40.398437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:19:40.398575 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:19:40.398682 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.398793 (+ 111us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.398869 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260924 21:19:40.400318 3831 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:19:40.401857 3831 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:19:40.408496 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.403958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40509 (local address 127.0.0.1:60964)
0924 21:19:40.404240 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:40.404280 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:40.404343 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:40.404590 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:40.404593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:40.404596 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:40.404663 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.404665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.405986 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.405987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.407059 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.407062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.408202 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.408203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.408307 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.408311 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:19:40.408361 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:19:40.408405 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":53,"threads_started":1}
I20260924 21:19:40.408557 4299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.403959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:38773 (local address 127.0.0.1:52748)
0924 21:19:40.404409 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:40.404421 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:40.404476 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:40.405346 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:40.405347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:40.405349 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:40.405392 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.405393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.406619 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.406620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.407591 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.407593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.408312 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.408312 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.408375 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.408378 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:19:40.408428 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:19:40.408460 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260924 21:19:40.408704 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.404235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38562 (local address 127.20.115.195:45245)
0924 21:19:40.404428 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.404432 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.404649 (+ 217us) server_negotiation.cc:407] Connection header received
0924 21:19:40.404820 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.404823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.404839 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.404907 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:40.405126 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.406053 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.407379 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.408038 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.408122 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.408167 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.408452 (+ 285us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.408490 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":97,"threads_started":1}
I20260924 21:19:40.408730 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.404143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:45245 (local address 127.0.0.1:38562)
0924 21:19:40.404596 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:40.404604 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:40.404708 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:40.404983 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:40.404983 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:40.404986 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:40.405033 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.405034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.406167 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.406168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.407293 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.407295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.408141 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.408142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.408221 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.408236 (+ 15us) client_negotiation.cc:769] Sending connection context
0924 21:19:40.408303 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 21:19:40.408354 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":37,"threads_started":1}
I20260924 21:19:40.408564 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.404083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52748 (local address 127.20.115.193:38773)
0924 21:19:40.405187 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.405191 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.405200 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:19:40.405235 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.405237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.405251 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.405365 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:40.405467 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.406495 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.407681 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.408227 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.408306 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.408358 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.408439 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.408482 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1067,"thread_start_us":91,"threads_started":1}
I20260924 21:19:40.409173 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.404133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60964 (local address 127.20.115.194:40509)
0924 21:19:40.404322 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.404327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.404363 (+ 36us) server_negotiation.cc:407] Connection header received
0924 21:19:40.404467 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.404470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.404486 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.404588 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:40.404730 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.405890 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.407171 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.408038 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.408112 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.408167 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.409049 (+ 882us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.409093 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":82,"threads_started":1}
I20260924 21:19:40.409523 4224 tablet_service.cc:1467] Processing CreateTablet for tablet c76fc97e4967451b87642285d5d673fd (DEFAULT_TABLE table=test-table [id=a3d8ba14f0354ea496b8bb7a99d59676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:40.409511 3956 tablet_service.cc:1467] Processing CreateTablet for tablet c76fc97e4967451b87642285d5d673fd (DEFAULT_TABLE table=test-table [id=a3d8ba14f0354ea496b8bb7a99d59676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:40.409788 4224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c76fc97e4967451b87642285d5d673fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:40.409830 3956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c76fc97e4967451b87642285d5d673fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:40.409766 4090 tablet_service.cc:1467] Processing CreateTablet for tablet c76fc97e4967451b87642285d5d673fd (DEFAULT_TABLE table=test-table [id=a3d8ba14f0354ea496b8bb7a99d59676]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:19:40.410035 4090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c76fc97e4967451b87642285d5d673fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:19:40.412218 4304 tablet_bootstrap.cc:492] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: Bootstrap starting.
I20260924 21:19:40.413021 4305 tablet_bootstrap.cc:492] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: Bootstrap starting.
I20260924 21:19:40.412963 4304 tablet_bootstrap.cc:654] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:40.413707 4305 tablet_bootstrap.cc:654] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:40.413744 4304 log.cc:824] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:40.414108 4306 tablet_bootstrap.cc:492] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Bootstrap starting.
I20260924 21:19:40.414314 4305 log.cc:824] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:40.414495 4304 tablet_bootstrap.cc:492] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: No bootstrap required, opened a new log
I20260924 21:19:40.414563 4304 ts_tablet_manager.cc:1397] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:19:40.414734 4306 tablet_bootstrap.cc:654] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Neither blocks nor log segments found. Creating new log.
I20260924 21:19:40.415093 4305 tablet_bootstrap.cc:492] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: No bootstrap required, opened a new log
I20260924 21:19:40.415150 4305 ts_tablet_manager.cc:1397] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:19:40.415304 4306 log.cc:824] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Log is configured to *not* fsync() on all Append() calls
I20260924 21:19:40.415979 4306 tablet_bootstrap.cc:492] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: No bootstrap required, opened a new log
I20260924 21:19:40.416077 4306 ts_tablet_manager.cc:1397] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:19:40.416548 4304 raft_consensus.cc:348] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:19:40.416731 4304 raft_consensus.cc:374] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:40.416767 4304 raft_consensus.cc:729] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c8c186b4f64b6686efa896ab96de40, State: Initialized, Role: FOLLOWER
I20260924 21:19:40.416950 4304 consensus_queue.cc:260] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:19:40.417114 4305 raft_consensus.cc:348] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:19:40.417234 4304 ts_tablet_manager.cc:1428] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 21:19:40.417245 4290 heartbeater.cc:503] Master 127.20.115.254:46259 was elected leader, sending a full tablet report...
I20260924 21:19:40.417284 4305 raft_consensus.cc:374] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:40.417321 4305 raft_consensus.cc:729] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce32df00534b4e29a67dd0c2a11b0374, State: Initialized, Role: FOLLOWER
I20260924 21:19:40.417510 4305 consensus_queue.cc:260] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:19:40.417806 4305 ts_tablet_manager.cc:1428] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 21:19:40.417816 4156 heartbeater.cc:503] Master 127.20.115.254:46259 was elected leader, sending a full tablet report...
I20260924 21:19:40.417855 4306 raft_consensus.cc:348] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:19:40.417997 4306 raft_consensus.cc:374] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:19:40.418030 4306 raft_consensus.cc:729] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117a8e037e7740818096e4852d0242dd, State: Initialized, Role: FOLLOWER
I20260924 21:19:40.418162 4306 consensus_queue.cc:260] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:19:40.418418 4306 ts_tablet_manager.cc:1428] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260924 21:19:40.418533 4022 heartbeater.cc:503] Master 127.20.115.254:46259 was elected leader, sending a full tablet report...
W20260924 21:19:40.471644 4157 tablet.cc:2369] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:19:40.568974 4023 tablet.cc:2369] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:19:40.626029 4291 tablet.cc:2369] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:19:40.720252 4312 raft_consensus.cc:482] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:19:40.720381 4312 raft_consensus.cc:504] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:19:40.720870 4312 leader_election.cc:288] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509), c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245)
I20260924 21:19:40.723990 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.721153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:42047 (local address 127.20.115.194:40509)
0924 21:19:40.721224 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.721226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.721257 (+ 31us) server_negotiation.cc:407] Connection header received
0924 21:19:40.721375 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.721377 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.721390 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.721421 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:40.721685 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.722206 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.723221 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.723635 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.723691 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.723697 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.723879 (+ 182us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.723926 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 21:19:40.723992 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.721072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40509 (local address 127.20.115.193:42047)
0924 21:19:40.721195 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:40.721207 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:40.721274 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:40.721510 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:40.721511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:40.721513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:40.721594 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.721596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.722284 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.722285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.723123 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.723125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.723720 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.723721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.723796 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.723800 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:19:40.723854 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 21:19:40.723885 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":5,"thread_start_us":81,"threads_started":1}
I20260924 21:19:40.724064 4303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.721325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:44439 (local address 127.20.115.195:45245)
0924 21:19:40.721375 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.721377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.721419 (+ 42us) server_negotiation.cc:407] Connection header received
0924 21:19:40.721489 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.721490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.721501 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.721553 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:19:40.721777 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.722372 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.723306 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.723704 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.723775 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.723784 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.723951 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.723993 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260924 21:19:40.724263 4314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.721267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:45245 (local address 127.20.115.193:44439)
0924 21:19:40.721344 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:40.721355 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:19:40.721446 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:19:40.721557 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:19:40.721557 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:19:40.721558 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:19:40.721675 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.721677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.722455 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.722456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.723226 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:19:40.723228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.723788 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.723789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:19:40.723870 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.723875 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:19:40.723931 (+ 56us) client_negotiation.cc:240] Negotiation successful
0924 21:19:40.723968 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":3,"thread_start_us":38,"threads_started":1}
I20260924 21:19:40.724414 4110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c76fc97e4967451b87642285d5d673fd" candidate_uuid: "117a8e037e7740818096e4852d0242dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce32df00534b4e29a67dd0c2a11b0374" is_pre_election: true
I20260924 21:19:40.724568 4110 raft_consensus.cc:2454] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117a8e037e7740818096e4852d0242dd in term 0.
I20260924 21:19:40.724612 4244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c76fc97e4967451b87642285d5d673fd" candidate_uuid: "117a8e037e7740818096e4852d0242dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c8c186b4f64b6686efa896ab96de40" is_pre_election: true
I20260924 21:19:40.724820 4244 raft_consensus.cc:2454] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 117a8e037e7740818096e4852d0242dd in term 0.
I20260924 21:19:40.724856 3912 leader_election.cc:302] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117a8e037e7740818096e4852d0242dd, ce32df00534b4e29a67dd0c2a11b0374; no voters:
I20260924 21:19:40.725008 4312 raft_consensus.cc:2781] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:19:40.725072 4312 raft_consensus.cc:482] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:19:40.725091 4312 raft_consensus.cc:3037] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:40.725817 4312 raft_consensus.cc:504] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:19:40.725911 4312 leader_election.cc:288] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 1 election: Requested vote from peers ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509), c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245)
I20260924 21:19:40.726068 4110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c76fc97e4967451b87642285d5d673fd" candidate_uuid: "117a8e037e7740818096e4852d0242dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce32df00534b4e29a67dd0c2a11b0374"
I20260924 21:19:40.726155 4110 raft_consensus.cc:3037] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:40.726126 4244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c76fc97e4967451b87642285d5d673fd" candidate_uuid: "117a8e037e7740818096e4852d0242dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c8c186b4f64b6686efa896ab96de40"
I20260924 21:19:40.726235 4244 raft_consensus.cc:3037] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:19:40.726955 4110 raft_consensus.cc:2454] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117a8e037e7740818096e4852d0242dd in term 1.
I20260924 21:19:40.727121 3912 leader_election.cc:302] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117a8e037e7740818096e4852d0242dd, ce32df00534b4e29a67dd0c2a11b0374; no voters:
I20260924 21:19:40.727136 4244 raft_consensus.cc:2454] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 117a8e037e7740818096e4852d0242dd in term 1.
I20260924 21:19:40.727263 4312 raft_consensus.cc:2781] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:19:40.727456 4312 raft_consensus.cc:686] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 1 LEADER]: Becoming Leader. State: Replica: 117a8e037e7740818096e4852d0242dd, State: Running, Role: LEADER
I20260924 21:19:40.727569 4312 consensus_queue.cc:237] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:19:40.728416 3830 catalog_manager.cc:5495] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 117a8e037e7740818096e4852d0242dd (127.20.115.193). New cstate: current_term: 1 leader_uuid: "117a8e037e7740818096e4852d0242dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } health_report { overall_health: HEALTHY } } }
I20260924 21:19:40.759922 4300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.757686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52758 (local address 127.20.115.193:38773)
0924 21:19:40.757763 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:19:40.757765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:19:40.757774 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:19:40.757808 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:19:40.757809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:19:40.757828 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:19:40.757892 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:19:40.758051 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.758509 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.759238 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:19:40.759421 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:19:40.759473 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:19:40.759602 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:19:40.759698 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:19:40.759758 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:19:40.759792 (+ 34us) server_negotiation.cc:299] Negotiation successful
0924 21:19:40.759809 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:19:40.761840 4244 raft_consensus.cc:1261] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [term 1 FOLLOWER]: Refusing update from remote peer 117a8e037e7740818096e4852d0242dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:40.762113 4110 raft_consensus.cc:1261] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [term 1 FOLLOWER]: Refusing update from remote peer 117a8e037e7740818096e4852d0242dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:19:40.762251 4315 consensus_queue.cc:1035] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:40.762408 4312 consensus_queue.cc:1035] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:19:40.765578 4320 mvcc.cc:205] Tried to move back new op lower bound from 7333006461997387776 to 7333006461860720640. Current Snapshot: MvccSnapshot[applied={T|T < 7333006461997387776}]
I20260924 21:19:40.767105 3831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:19:40.767212 3831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:19:40.768005 3831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Sending DeleteTablet for 3 replicas of tablet c76fc97e4967451b87642285d5d673fd
I20260924 21:19:40.768373 3956 tablet_service.cc:1514] Processing DeleteTablet for tablet c76fc97e4967451b87642285d5d673fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:40 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:52748
I20260924 21:19:40.768618 4327 tablet_replica.cc:330] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: stopping tablet replica
I20260924 21:19:40.768632 4090 tablet_service.cc:1514] Processing DeleteTablet for tablet c76fc97e4967451b87642285d5d673fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:40 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:60964
I20260924 21:19:40.768728 4327 raft_consensus.cc:2229] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:19:40.768874 4328 tablet_replica.cc:330] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: stopping tablet replica
I20260924 21:19:40.768889 4327 raft_consensus.cc:2258] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:40.768885 4224 tablet_service.cc:1514] Processing DeleteTablet for tablet c76fc97e4967451b87642285d5d673fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:40 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:38562
I20260924 21:19:40.769083 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 3893
I20260924 21:19:40.769140 4328 raft_consensus.cc:2229] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:19:40.769244 4328 raft_consensus.cc:2258] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:40.769637 4329 tablet_replica.cc:330] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: stopping tablet replica
I20260924 21:19:40.769724 4328 ts_tablet_manager.cc:1905] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:19:40.769737 4329 raft_consensus.cc:2229] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:19:40.769836 4329 raft_consensus.cc:2258] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:19:40.771276 4329 ts_tablet_manager.cc:1905] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:19:40.771867 4328 ts_tablet_manager.cc:1918] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:19:40.771947 4328 log.cc:1196] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/c76fc97e4967451b87642285d5d673fd
I20260924 21:19:40.772213 4328 ts_tablet_manager.cc:1939] T c76fc97e4967451b87642285d5d673fd P ce32df00534b4e29a67dd0c2a11b0374: Deleting consensus metadata
I20260924 21:19:40.772854 3818 catalog_manager.cc:4841] TS ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509): tablet c76fc97e4967451b87642285d5d673fd (table test-table [id=a3d8ba14f0354ea496b8bb7a99d59676]) successfully deleted
I20260924 21:19:40.773083 4329 ts_tablet_manager.cc:1918] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:19:40.773162 4329 log.cc:1196] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/c76fc97e4967451b87642285d5d673fd
I20260924 21:19:40.773423 4329 ts_tablet_manager.cc:1939] T c76fc97e4967451b87642285d5d673fd P c9c8c186b4f64b6686efa896ab96de40: Deleting consensus metadata
I20260924 21:19:40.773981 3818 catalog_manager.cc:4841] TS c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245): tablet c76fc97e4967451b87642285d5d673fd (table test-table [id=a3d8ba14f0354ea496b8bb7a99d59676]) successfully deleted
W20260924 21:19:40.775445 3815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:38773 (error 108)
I20260924 21:19:40.775652 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 4027
I20260924 21:19:40.776466 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:19:40.775812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:38773 (local address 127.0.0.1:52760)
0924 21:19:40.776385 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0924 21:19:40.776425 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553}
W20260924 21:19:40.776605 3815 catalog_manager.cc:4584] TS 117a8e037e7740818096e4852d0242dd (127.20.115.193:38773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c76fc97e4967451b87642285d5d673fd: Network error: Client connection negotiation failed: client connection to 127.20.115.193:38773: connect: Connection refused (error 111)
I20260924 21:19:40.780448 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 4161
I20260924 21:19:40.784695 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 3799
I20260924 21:20:00.790233 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46259
--webserver_interface=127.20.115.254
--webserver_port=34215
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:46259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:00.871176 4331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:00.871372 4331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:00.871393 4331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:00.872459 4331 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 21:20:00.872490 4331 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 21:20:00.872503 4331 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 21:20:00.872514 4331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:00.872527 4331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:00.872540 4331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:00.872552 4331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:00.872565 4331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:00.874039 4331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:46259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=34215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:00.874221 4331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:00.874382 4331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:00.882805 4337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:00.882833 4331 server_base.cc:1034] running on GCE node
W20260924 21:20:00.882771 4340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:00.882783 4338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:00.883332 4331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:00.883680 4331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:00.884845 4331 hybrid_clock.cc:648] HybridClock initialized: now 1790284800884823 us; error 50 us; skew 500 ppm
Sep 24 21:20:00 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284800, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:00.887265 4331 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:20:00.887701 4331 webserver.cc:460] Webserver started at http://127.20.115.254:34215/ using document root <none> and password file <none>
I20260924 21:20:00.887934 4331 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:00.888018 4331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:00.889500 4331 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:20:00.890134 4347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:00.890311 4331 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:20:00.890417 4331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "80cba6bf686646d28a3e4fdbb6607fa3"
format_stamp: "Formatted at 2026-09-24 21:19:39 on dist-test-slave-cxpd"
server_key: "8ccd1ae2bf31de35f6ce6f87c49e3518"
server_key_iv: "875b81a36dd3e73f6ece38605cb5c614"
server_key_version: "encryptionkey@0"
I20260924 21:20:00.890838 4331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:00.904316 4331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:00.906291 4331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:00.906437 4331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:00.910499 4331 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:46259
I20260924 21:20:00.910529 4399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:46259 every 8 connection(s)
I20260924 21:20:00.910902 4331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:00.912331 4400 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:20:00.913072 4400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Bootstrap starting.
I20260924 21:20:00.915549 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 4331
I20260924 21:20:00.915874 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:38773
--local_ip_for_outbound_sockets=127.20.115.193
--tserver_master_addrs=127.20.115.254:46259
--webserver_port=36697
--webserver_interface=127.20.115.193
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:20:00.921957 4400 log.cc:824] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:00.923764 4400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:20:00.924058 4400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Bootstrap complete.
I20260924 21:20:00.926307 4400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:20:00.926541 4400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80cba6bf686646d28a3e4fdbb6607fa3, State: Initialized, Role: FOLLOWER
I20260924 21:20:00.926738 4400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:20:00.926832 4400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:00.926889 4400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:00.926936 4400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 1 FOLLOWER]: Advancing to term 2
I20260924 21:20:00.927879 4400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:20:00.928022 4400 leader_election.cc:302] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80cba6bf686646d28a3e4fdbb6607fa3; no voters:
I20260924 21:20:00.928229 4400 leader_election.cc:288] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260924 21:20:00.928262 4404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 2 FOLLOWER]: Leader election won for term 2
I20260924 21:20:00.928370 4404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 2 LEADER]: Becoming Leader. State: Replica: 80cba6bf686646d28a3e4fdbb6607fa3, State: Running, Role: LEADER
I20260924 21:20:00.928488 4400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:00.928462 4404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:20:00.928740 4404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80cba6bf686646d28a3e4fdbb6607fa3. Latest consensus state: current_term: 2 leader_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } } }
I20260924 21:20:00.928891 4404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:00.928740 4405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } } }
I20260924 21:20:00.929176 4405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:00.929308 4409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:00.930058 4409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:00.930161 4409 catalog_manager.cc:1259] Loaded cluster ID: 4804670773f74f50a65f73e792a44b67
I20260924 21:20:00.930193 4409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:00.931628 4409 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:00.931810 4409 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Loaded TSK: 0
I20260924 21:20:00.949934 4409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Generated new TSK 1
I20260924 21:20:00.950030 4409 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 21:20:01.002213 4402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:01.002398 4402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:01.002419 4402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:01.003545 4402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:01.003584 4402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:01.003610 4402 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:20:01.005033 4402 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:38773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=36697
--tserver_master_addrs=127.20.115.254:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:01.005249 4402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:01.005486 4402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:01.011226 4427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:01.011305 4430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:01.011442 4402 server_base.cc:1034] running on GCE node
W20260924 21:20:01.011305 4428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:01.011849 4402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:01.012116 4402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:01.013295 4402 hybrid_clock.cc:648] HybridClock initialized: now 1790284801013276 us; error 32 us; skew 500 ppm
Sep 24 21:20:01 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284801, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:01.016201 4402 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:20:01.016672 4402 webserver.cc:460] Webserver started at http://127.20.115.193:36697/ using document root <none> and password file <none>
I20260924 21:20:01.016937 4402 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:01.017021 4402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:01.019454 4402 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260924 21:20:01.020237 4437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:01.020418 4402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:01.020532 4402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "117a8e037e7740818096e4852d0242dd"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "533a9481ecbe7b9ab3af901a36ec879d"
server_key_iv: "a86592a9788fae9c55936538eb50d48d"
server_key_version: "encryptionkey@0"
I20260924 21:20:01.020931 4402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:01.031095 4402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:01.033329 4402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:01.033488 4402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:01.033720 4402 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:01.034129 4444 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:20:01.035957 4402 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:20:01.036031 4402 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:20:01.036074 4402 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:20:01.036703 4402 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:20:01.036761 4402 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260924 21:20:01.036854 4444 tablet_bootstrap.cc:492] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Bootstrap starting.
I20260924 21:20:01.043901 4402 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:38773
I20260924 21:20:01.044247 4402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:20:01.047752 4444 log.cc:824] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:01.048432 4551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:38773 every 8 connection(s)
I20260924 21:20:01.050534 4444 tablet_bootstrap.cc:492] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:20:01.050812 4444 tablet_bootstrap.cc:492] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Bootstrap complete.
I20260924 21:20:01.051070 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 4402
I20260924 21:20:01.051342 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:40509
--local_ip_for_outbound_sockets=127.20.115.194
--tserver_master_addrs=127.20.115.254:46259
--webserver_port=38607
--webserver_interface=127.20.115.194
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:20:01.052002 4444 ts_tablet_manager.cc:1397] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260924 21:20:01.053406 4444 raft_consensus.cc:348] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.053746 4444 raft_consensus.cc:729] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117a8e037e7740818096e4852d0242dd, State: Initialized, Role: FOLLOWER
I20260924 21:20:01.053876 4444 consensus_queue.cc:260] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.054160 4444 ts_tablet_manager.cc:1428] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Sep 24 21:20:01 dist-test-slave-cxpd krb5kdc[3786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284801, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:01.059094 4554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.044939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:44257)
0924 21:20:01.045125 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:01.045166 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:01.045601 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:01.048632 (+ 3031us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:01.048636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:01.049067 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:01.049326 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.049331 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.051055 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.051056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.052388 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.052391 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.053309 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.053310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.053434 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.054098 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:01.054107 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:01.055533 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:01.057499 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:01.057505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:01.057507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:01.057726 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:01.058015 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:01.058016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:01.058016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:01.058122 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:01.058491 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:01.058496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:01.058628 (+ 132us) client_negotiation.cc:769] Sending connection context
0924 21:20:01.058735 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 21:20:01.058901 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":115,"threads_started":1}
I20260924 21:20:01.059094 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.045015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:44257 (local address 127.20.115.254:46259)
0924 21:20:01.048126 (+ 3111us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.048135 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.048177 (+ 42us) server_negotiation.cc:407] Connection header received
0924 21:20:01.048301 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.048306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.048348 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.049438 (+ 1090us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:01.049525 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.050912 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.052486 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.053116 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.053222 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.055649 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:01.055653 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:01.055653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:01.055664 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:01.057395 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.057825 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.057828 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.057828 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.057927 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.058189 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.058190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.058190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.058408 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:01.058504 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.058791 (+ 287us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.058912 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3039,"thread_start_us":75,"threads_started":1}
I20260924 21:20:01.059973 4552 heartbeater.cc:347] Connected to a master server at 127.20.115.254:46259
I20260924 21:20:01.060091 4552 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:01.060253 4552 heartbeater.cc:511] Master 127.20.115.254:46259 requested a full tablet report, sending...
I20260924 21:20:01.060988 4364 ts_manager.cc:194] Registered new tserver with Master: 117a8e037e7740818096e4852d0242dd (127.20.115.193:38773)
I20260924 21:20:01.062083 4364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:44257
I20260924 21:20:01.062884 4552 heartbeater.cc:503] Master 127.20.115.254:46259 was elected leader, sending a full tablet report...
I20260924 21:20:01.067909 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.061784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.20.115.193:38773)
0924 21:20:01.065111 (+ 3327us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.065114 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.065122 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:20:01.065146 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.065148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.065160 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.065266 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:01.065414 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.066122 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.067097 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.067545 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.067630 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.067681 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.067807 (+ 126us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.067841 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3302,"thread_start_us":65,"threads_started":1}
I20260924 21:20:01.067909 4559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.061565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:38773 (local address 127.0.0.1:51202)
0924 21:20:01.064918 (+ 3353us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:01.064948 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:01.065009 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:01.065275 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:01.065278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:01.065280 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:01.065338 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.065340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.066209 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.066210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.067006 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.067009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.067628 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.067629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.067707 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.067719 (+ 12us) client_negotiation.cc:769] Sending connection context
0924 21:20:01.067789 (+ 70us) client_negotiation.cc:240] Negotiation successful
0924 21:20:01.067841 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3317,"thread_start_us":81,"threads_started":1}
I20260924 21:20:01.068456 4469 tablet_service.cc:1514] Processing DeleteTablet for tablet c76fc97e4967451b87642285d5d673fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:19:40 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:51202
I20260924 21:20:01.068976 4561 tablet_replica.cc:330] stopping tablet replica
I20260924 21:20:01.069100 4561 raft_consensus.cc:2229] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:01.069167 4561 raft_consensus.cc:2258] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:01.069455 4561 ts_tablet_manager.cc:1905] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:01.071560 4561 ts_tablet_manager.cc:1918] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:01.071645 4561 log.cc:1196] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal/wals/c76fc97e4967451b87642285d5d673fd
I20260924 21:20:01.071858 4561 ts_tablet_manager.cc:1939] T c76fc97e4967451b87642285d5d673fd P 117a8e037e7740818096e4852d0242dd: Deleting consensus metadata
I20260924 21:20:01.072376 4348 catalog_manager.cc:4841] TS 117a8e037e7740818096e4852d0242dd (127.20.115.193:38773): tablet c76fc97e4967451b87642285d5d673fd (table test-table [id=a3d8ba14f0354ea496b8bb7a99d59676]) successfully deleted
W20260924 21:20:01.137025 4557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:01.137213 4557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:01.137238 4557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:01.138386 4557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:01.138427 4557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:01.138453 4557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:20:01.139916 4557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:40509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=38607
--tserver_master_addrs=127.20.115.254:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:01.140175 4557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:01.140398 4557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:01.145246 4567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:01.145331 4569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:01.145283 4566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:01.145524 4557 server_base.cc:1034] running on GCE node
I20260924 21:20:01.145720 4557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:01.145961 4557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:01.147115 4557 hybrid_clock.cc:648] HybridClock initialized: now 1790284801147121 us; error 52 us; skew 500 ppm
Sep 24 21:20:01 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284801, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:01.149574 4557 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:20:01.149940 4557 webserver.cc:460] Webserver started at http://127.20.115.194:38607/ using document root <none> and password file <none>
I20260924 21:20:01.150120 4557 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:01.150170 4557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:01.151407 4557 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:20:01.151928 4576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:01.152137 4557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:01.152225 4557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "ce32df00534b4e29a67dd0c2a11b0374"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "a638a2670610d8c7cffdd647a1acddda"
server_key_iv: "ecc7610177ca6f074035603e529633e0"
server_key_version: "encryptionkey@0"
I20260924 21:20:01.152534 4557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:01.164614 4557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:01.166420 4557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:01.166572 4557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:01.166808 4557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:01.167115 4557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:01.167168 4557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:01.167201 4557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:01.167229 4557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:01.173154 4557 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:40509
I20260924 21:20:01.173249 4689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:40509 every 8 connection(s)
I20260924 21:20:01.173555 4557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:20:01.177484 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 4557
I20260924 21:20:01.177799 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:45245
--local_ip_for_outbound_sockets=127.20.115.195
--tserver_master_addrs=127.20.115.254:46259
--webserver_port=44731
--webserver_interface=127.20.115.195
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 24 21:20:01 dist-test-slave-cxpd krb5kdc[3786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284801, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:01.185699 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.174289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:34731)
0924 21:20:01.174455 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:01.174490 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:01.175001 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:01.175349 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:01.175353 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:01.176173 (+ 820us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:01.176445 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.176451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.177624 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.177625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.179058 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.179061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.179631 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.179632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.179780 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.180333 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:01.180343 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:01.182406 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:01.184296 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:01.184302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:01.184303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:01.184507 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:01.184765 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:01.184765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:01.184765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:01.184925 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:01.185238 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:01.185243 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:01.185350 (+ 107us) client_negotiation.cc:769] Sending connection context
0924 21:20:01.185431 (+ 81us) client_negotiation.cc:240] Negotiation successful
0924 21:20:01.185544 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260924 21:20:01.185746 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.174281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:34731 (local address 127.20.115.254:46259)
0924 21:20:01.174333 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.174336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.174576 (+ 240us) server_negotiation.cc:407] Connection header received
0924 21:20:01.175164 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.175166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.175183 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.175246 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:01.176565 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.177484 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.179161 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.179540 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.179626 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.182493 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:01.182495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:01.182496 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:01.182505 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:01.184222 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.184629 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.184631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.184631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.184691 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.184998 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.184999 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.185000 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.185162 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:01.185247 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.185482 (+ 235us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.185590 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:20:01.186209 4690 heartbeater.cc:347] Connected to a master server at 127.20.115.254:46259
I20260924 21:20:01.186321 4690 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:01.186527 4690 heartbeater.cc:511] Master 127.20.115.254:46259 requested a full tablet report, sending...
I20260924 21:20:01.186970 4364 ts_manager.cc:194] Registered new tserver with Master: ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509)
I20260924 21:20:01.187337 4364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:34731
W20260924 21:20:01.262513 4693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:01.262696 4693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:01.262718 4693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:01.263832 4693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:01.263873 4693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:01.263899 4693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:20:01.265321 4693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:45245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=44731
--tserver_master_addrs=127.20.115.254:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:01.265549 4693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:01.265806 4693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:01.270776 4698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:01.270819 4699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:01.270792 4701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:01.270952 4693 server_base.cc:1034] running on GCE node
I20260924 21:20:01.271240 4693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:01.271499 4693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:01.272699 4693 hybrid_clock.cc:648] HybridClock initialized: now 1790284801272671 us; error 38 us; skew 500 ppm
Sep 24 21:20:01 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284801, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:01.275233 4693 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:20:01.275641 4693 webserver.cc:460] Webserver started at http://127.20.115.195:44731/ using document root <none> and password file <none>
I20260924 21:20:01.275871 4693 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:01.275954 4693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:01.277277 4693 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:01.277861 4708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:01.278018 4693 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:01.278117 4693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "c9c8c186b4f64b6686efa896ab96de40"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "085058bd3a211f4e13096e29747d7b2e"
server_key_iv: "00ec6ae40b26b468e64c815bb8aa77c6"
server_key_version: "encryptionkey@0"
I20260924 21:20:01.278430 4693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:01.289918 4693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:01.292207 4693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:01.292361 4693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:01.292589 4693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:01.292959 4693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:01.293011 4693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:01.293053 4693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:01.293081 4693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:01.299012 4693 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:45245
I20260924 21:20:01.299130 4821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:45245 every 8 connection(s)
I20260924 21:20:01.299417 4693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
Sep 24 21:20:01 dist-test-slave-cxpd krb5kdc[3786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284801, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:01.306325 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 4693
I20260924 21:20:01.308686 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.300095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:38701 (local address 127.20.115.254:46259)
0924 21:20:01.300188 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.300191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.300365 (+ 174us) server_negotiation.cc:407] Connection header received
0924 21:20:01.300969 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.300970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.300984 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.301029 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:01.301948 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.302445 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.303428 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.303644 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.303719 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.305697 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:01.305698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:01.305699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:01.305706 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:01.307254 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.307673 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.307674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.307675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.307738 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.307993 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.307994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.307994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.308163 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:01.308245 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.308519 (+ 274us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.308598 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 21:20:01.308900 4824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.300061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:38701)
0924 21:20:01.300243 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:01.300278 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:01.300846 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:01.301111 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:01.301115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:01.301663 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:01.301845 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.301849 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.302538 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.302540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.303333 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.303335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.303736 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.303737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.303854 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.304282 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:01.304292 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:01.305601 (+ 1309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:01.307355 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:01.307361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:01.307363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:01.307592 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:01.307810 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:01.307810 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:01.307811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:01.307914 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:01.308237 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:01.308243 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:01.308376 (+ 133us) client_negotiation.cc:769] Sending connection context
0924 21:20:01.308484 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 21:20:01.308644 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
I20260924 21:20:01.309396 4822 heartbeater.cc:347] Connected to a master server at 127.20.115.254:46259
I20260924 21:20:01.309508 4822 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:01.309703 4822 heartbeater.cc:511] Master 127.20.115.254:46259 requested a full tablet report, sending...
I20260924 21:20:01.310075 4364 ts_manager.cc:194] Registered new tserver with Master: c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245)
I20260924 21:20:01.310539 4364 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:38701
I20260924 21:20:01.313345 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.306964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36154 (local address 127.20.115.254:46259)
0924 21:20:01.307114 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.307116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.307168 (+ 52us) server_negotiation.cc:407] Connection header received
0924 21:20:01.307262 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.307263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.307276 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.307351 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:01.307935 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.308438 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.309496 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.309764 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.309834 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.311070 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:01.311072 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:01.311072 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:01.311079 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:01.312224 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.312553 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.312554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.312554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.312608 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.312831 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.312831 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.312832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.312939 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:01.313130 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.313168 (+ 38us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.313246 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260924 21:20:01.314045 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:20:01.316359 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.314337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.20.115.254:46259)
0924 21:20:01.314370 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.314372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.314462 (+ 90us) server_negotiation.cc:407] Connection header received
0924 21:20:01.314513 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.314514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.314538 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.314598 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:20:01.314688 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.315117 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.315813 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.315989 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.316035 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.316182 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:20:01.316216 (+ 34us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0924 21:20:01.316324 (+ 108us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36164: token expired
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 21:20:01.316422 4826 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36164: token expired
W20260924 21:20:01.316557 4294 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.115.254:46259)
I20260924 21:20:01.322388 4826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.316838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36166 (local address 127.20.115.254:46259)
0924 21:20:01.316867 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.316868 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.316875 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:20:01.316897 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.316897 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.316905 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.316957 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:01.317419 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.317850 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.318559 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.318730 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.318785 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.319953 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:01.319954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:01.319954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:01.319960 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:01.321384 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.321712 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.321713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.321713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.321765 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:01.321957 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:01.321958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:01.321959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:01.322092 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:01.322167 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.322280 (+ 113us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.322338 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260924 21:20:01.323485 4364 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:20:01.325155 4364 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:20:01.327030 4469 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e (DEFAULT_TABLE table=test-table [id=493b0bbb7bd34e438cf8af9213da7e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:01.327255 4469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:01.330000 4444 tablet_bootstrap.cc:492] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Bootstrap starting.
I20260924 21:20:01.330320 4444 tablet_bootstrap.cc:654] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:01.330758 4559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.326802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:45245 (local address 127.0.0.1:45010)
0924 21:20:01.327030 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:01.327040 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:01.327083 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:01.327433 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:01.327434 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:01.327436 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:01.327475 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.327476 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.328847 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.328848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.329869 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.329871 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.330537 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.330538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.330600 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.330603 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:01.330654 (+ 51us) client_negotiation.cc:240] Negotiation successful
0924 21:20:01.330697 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213}
I20260924 21:20:01.330884 4444 tablet_bootstrap.cc:492] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: No bootstrap required, opened a new log
I20260924 21:20:01.330929 4444 ts_tablet_manager.cc:1397] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:01.330977 4829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.326882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45010 (local address 127.20.115.195:45245)
0924 21:20:01.327247 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.327252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.327262 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:20:01.327299 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.327301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.327317 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.327428 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:01.327542 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.328756 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.329958 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.330434 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.330537 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.330607 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.330860 (+ 253us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.330901 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":111,"threads_started":1}
I20260924 21:20:01.331034 4444 raft_consensus.cc:348] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.331104 4444 raft_consensus.cc:374] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:01.331152 4444 raft_consensus.cc:729] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117a8e037e7740818096e4852d0242dd, State: Initialized, Role: FOLLOWER
I20260924 21:20:01.331211 4444 consensus_queue.cc:260] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.331295 4444 ts_tablet_manager.cc:1428] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:01.331723 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.326891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40509 (local address 127.0.0.1:35158)
0924 21:20:01.327059 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:01.327070 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:01.327121 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:01.327604 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:01.327604 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:01.327606 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:01.327640 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.327641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.328642 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.328643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.329872 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.329874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.331472 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.331473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.331535 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.331539 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:01.331606 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 21:20:01.331652 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":103,"threads_started":1}
I20260924 21:20:01.331733 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.327246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35158 (local address 127.20.115.194:40509)
0924 21:20:01.327446 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.327450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.327470 (+ 20us) server_negotiation.cc:407] Connection header received
0924 21:20:01.327508 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.327510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.327526 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.327608 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:01.327694 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.328561 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.330645 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.331379 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.331470 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.331526 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.331619 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.331658 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":99,"threads_started":1}
I20260924 21:20:01.332247 4756 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e (DEFAULT_TABLE table=test-table [id=493b0bbb7bd34e438cf8af9213da7e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:01.332425 4624 tablet_service.cc:1467] Processing CreateTablet for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e (DEFAULT_TABLE table=test-table [id=493b0bbb7bd34e438cf8af9213da7e69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:01.332623 4756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:01.332760 4624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:01.335287 4832 tablet_bootstrap.cc:492] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: Bootstrap starting.
I20260924 21:20:01.335417 4833 tablet_bootstrap.cc:492] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: Bootstrap starting.
I20260924 21:20:01.335935 4832 tablet_bootstrap.cc:654] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:01.335944 4833 tablet_bootstrap.cc:654] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:01.336401 4833 log.cc:824] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:01.336431 4832 log.cc:824] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:01.337105 4833 tablet_bootstrap.cc:492] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: No bootstrap required, opened a new log
I20260924 21:20:01.337162 4832 tablet_bootstrap.cc:492] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: No bootstrap required, opened a new log
I20260924 21:20:01.337184 4833 ts_tablet_manager.cc:1397] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:20:01.337245 4832 ts_tablet_manager.cc:1397] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:20:01.338614 4833 raft_consensus.cc:348] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.338645 4832 raft_consensus.cc:348] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.338793 4833 raft_consensus.cc:374] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:01.338798 4832 raft_consensus.cc:374] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:01.338842 4833 raft_consensus.cc:729] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce32df00534b4e29a67dd0c2a11b0374, State: Initialized, Role: FOLLOWER
I20260924 21:20:01.338867 4832 raft_consensus.cc:729] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c8c186b4f64b6686efa896ab96de40, State: Initialized, Role: FOLLOWER
I20260924 21:20:01.338971 4833 consensus_queue.cc:260] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.338979 4832 consensus_queue.cc:260] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.339258 4690 heartbeater.cc:503] Master 127.20.115.254:46259 was elected leader, sending a full tablet report...
I20260924 21:20:01.339274 4832 ts_tablet_manager.cc:1428] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:01.339303 4822 heartbeater.cc:503] Master 127.20.115.254:46259 was elected leader, sending a full tablet report...
I20260924 21:20:01.339545 4833 ts_tablet_manager.cc:1428] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:01.389449 4836 raft_consensus.cc:482] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:01.389569 4836 raft_consensus.cc:504] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.389993 4836 leader_election.cc:288] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245), 117a8e037e7740818096e4852d0242dd (127.20.115.193:38773)
I20260924 21:20:01.392843 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.390173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:38773 (local address 127.20.115.194:50761)
0924 21:20:01.390240 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:01.390253 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:01.390308 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:01.390475 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:01.390476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:01.390478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:01.390546 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.390548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.391183 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.391184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.391937 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.391939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.392527 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.392528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.392637 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.392643 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 21:20:01.392707 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 21:20:01.392745 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260924 21:20:01.392894 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.390306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:50761 (local address 127.20.115.193:38773)
0924 21:20:01.390352 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.390354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.390363 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:20:01.390392 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.390393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.390403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.390456 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:01.390623 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.391101 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.392033 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.392451 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.392525 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.392534 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.392726 (+ 192us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.392768 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:20:01.393133 4829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.390301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:49389 (local address 127.20.115.195:45245)
0924 21:20:01.390423 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.390425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.390516 (+ 91us) server_negotiation.cc:407] Connection header received
0924 21:20:01.390577 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.390578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.390586 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.390660 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:01.390809 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.391277 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.392210 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.392667 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.392761 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.392769 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.393019 (+ 250us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.393075 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260924 21:20:01.393131 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.390172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:45245 (local address 127.20.115.194:49389)
0924 21:20:01.390438 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:01.390448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:01.390523 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:01.390661 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:01.390662 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:01.390664 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:01.390739 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.390740 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.391359 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.391360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.392117 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:01.392119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.392764 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.392765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:01.392882 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.392885 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:01.393005 (+ 120us) client_negotiation.cc:240] Negotiation successful
0924 21:20:01.393056 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":80,"thread_start_us":113,"threads_started":1}
I20260924 21:20:01.393126 4506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec02675c74a4e9ab0fe7ffebbd8c22e" candidate_uuid: "ce32df00534b4e29a67dd0c2a11b0374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117a8e037e7740818096e4852d0242dd" is_pre_election: true
I20260924 21:20:01.393285 4506 raft_consensus.cc:2454] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce32df00534b4e29a67dd0c2a11b0374 in term 0.
I20260924 21:20:01.393503 4577 leader_election.cc:302] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 117a8e037e7740818096e4852d0242dd, ce32df00534b4e29a67dd0c2a11b0374; no voters:
I20260924 21:20:01.393460 4776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec02675c74a4e9ab0fe7ffebbd8c22e" candidate_uuid: "ce32df00534b4e29a67dd0c2a11b0374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c8c186b4f64b6686efa896ab96de40" is_pre_election: true
I20260924 21:20:01.393625 4776 raft_consensus.cc:2454] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce32df00534b4e29a67dd0c2a11b0374 in term 0.
I20260924 21:20:01.393781 4836 raft_consensus.cc:2781] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:20:01.393847 4836 raft_consensus.cc:482] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:20:01.393882 4836 raft_consensus.cc:3037] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:01.394619 4836 raft_consensus.cc:504] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.394754 4836 leader_election.cc:288] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [CANDIDATE]: Term 1 election: Requested vote from peers c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245), 117a8e037e7740818096e4852d0242dd (127.20.115.193:38773)
I20260924 21:20:01.394898 4776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec02675c74a4e9ab0fe7ffebbd8c22e" candidate_uuid: "ce32df00534b4e29a67dd0c2a11b0374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9c8c186b4f64b6686efa896ab96de40"
I20260924 21:20:01.394984 4776 raft_consensus.cc:3037] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:01.394923 4506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ec02675c74a4e9ab0fe7ffebbd8c22e" candidate_uuid: "ce32df00534b4e29a67dd0c2a11b0374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "117a8e037e7740818096e4852d0242dd"
I20260924 21:20:01.395033 4506 raft_consensus.cc:3037] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:01.395686 4776 raft_consensus.cc:2454] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce32df00534b4e29a67dd0c2a11b0374 in term 1.
I20260924 21:20:01.395861 4580 leader_election.cc:302] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9c8c186b4f64b6686efa896ab96de40, ce32df00534b4e29a67dd0c2a11b0374; no voters:
I20260924 21:20:01.395978 4836 raft_consensus.cc:2781] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:01.396124 4836 raft_consensus.cc:686] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 1 LEADER]: Becoming Leader. State: Replica: ce32df00534b4e29a67dd0c2a11b0374, State: Running, Role: LEADER
I20260924 21:20:01.396229 4836 consensus_queue.cc:237] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:01.396279 4506 raft_consensus.cc:2454] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce32df00534b4e29a67dd0c2a11b0374 in term 1.
I20260924 21:20:01.397099 4364 catalog_manager.cc:5495] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194). New cstate: current_term: 1 leader_uuid: "ce32df00534b4e29a67dd0c2a11b0374" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } health_report { overall_health: UNKNOWN } } }
I20260924 21:20:01.403241 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.401148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35172 (local address 127.20.115.194:40509)
0924 21:20:01.401207 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:01.401208 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:01.401233 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:20:01.401258 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:01.401259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:01.401273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:01.401321 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:20:01.401417 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.401842 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.402560 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:01.402735 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:01.402788 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:01.402912 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:20:01.402996 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:20:01.403061 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:01.403119 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 21:20:01.403133 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:20:01.404913 4776 raft_consensus.cc:1261] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [term 1 FOLLOWER]: Refusing update from remote peer ce32df00534b4e29a67dd0c2a11b0374: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:01.405001 4506 raft_consensus.cc:1261] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Refusing update from remote peer ce32df00534b4e29a67dd0c2a11b0374: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:01.405201 4836 consensus_queue.cc:1035] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:01.405310 4839 consensus_queue.cc:1035] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [LEADER]: Connected to new peer: Peer: permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:01.407454 4842 mvcc.cc:205] Tried to move back new op lower bound from 7333006546551869440 to 7333006546519830528. Current Snapshot: MvccSnapshot[applied={T|T < 7333006546551869440}]
I20260924 21:20:01.408345 4843 mvcc.cc:205] Tried to move back new op lower bound from 7333006546551869440 to 7333006546519830528. Current Snapshot: MvccSnapshot[applied={T|T < 7333006546551869440 or (T in {7333006546551869440})}]
I20260924 21:20:01.408485 4844 mvcc.cc:205] Tried to move back new op lower bound from 7333006546551869440 to 7333006546519830528. Current Snapshot: MvccSnapshot[applied={T|T < 7333006546551869440 or (T in {7333006546551869440})}]
I20260924 21:20:01.409997 4364 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:01.410096 4364 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:01.410799 4364 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Sending DeleteTablet for 3 replicas of tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e
I20260924 21:20:01.411103 4469 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:01 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:51202
I20260924 21:20:01.411132 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 4402
I20260924 21:20:01.411134 4756 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:01 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:45010
I20260924 21:20:01.411194 4561 tablet_replica.cc:330] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: stopping tablet replica
I20260924 21:20:01.411244 4561 raft_consensus.cc:2229] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:01.411288 4561 raft_consensus.cc:2258] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:01.415459 4624 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:01 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:35158
I20260924 21:20:01.415690 4851 tablet_replica.cc:330] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: stopping tablet replica
I20260924 21:20:01.415691 4852 tablet_replica.cc:330] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: stopping tablet replica
I20260924 21:20:01.415791 4851 raft_consensus.cc:2229] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:01.415870 4852 raft_consensus.cc:2229] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:20:01.415894 4851 raft_consensus.cc:2258] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:01.416029 4852 raft_consensus.cc:2258] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:01.416453 4852 ts_tablet_manager.cc:1905] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:01.417030 4851 ts_tablet_manager.cc:1905] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:01.418138 4852 ts_tablet_manager.cc:1918] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:01.418215 4852 log.cc:1196] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/0ec02675c74a4e9ab0fe7ffebbd8c22e
I20260924 21:20:01.418500 4852 ts_tablet_manager.cc:1939] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P ce32df00534b4e29a67dd0c2a11b0374: Deleting consensus metadata
I20260924 21:20:01.419001 4351 catalog_manager.cc:4841] TS ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509): tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e (table test-table [id=493b0bbb7bd34e438cf8af9213da7e69]) successfully deleted
I20260924 21:20:01.419178 4851 ts_tablet_manager.cc:1918] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:01.419251 4851 log.cc:1196] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/0ec02675c74a4e9ab0fe7ffebbd8c22e
W20260924 21:20:01.419502 4348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:38773 (error 108)
I20260924 21:20:01.419679 4851 ts_tablet_manager.cc:1939] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P c9c8c186b4f64b6686efa896ab96de40: Deleting consensus metadata
I20260924 21:20:01.419811 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 4557
I20260924 21:20:01.420055 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:01.419922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:38773 (local address 127.0.0.1:51210)
0924 21:20:01.419987 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:01.420026 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:38773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260924 21:20:01.420257 4348 catalog_manager.cc:4584] TS 117a8e037e7740818096e4852d0242dd (127.20.115.193:38773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e: Network error: Client connection negotiation failed: client connection to 127.20.115.193:38773: connect: Connection refused (error 111)
I20260924 21:20:01.421531 4351 catalog_manager.cc:4841] TS c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245): tablet 0ec02675c74a4e9ab0fe7ffebbd8c22e (table test-table [id=493b0bbb7bd34e438cf8af9213da7e69]) successfully deleted
I20260924 21:20:01.424474 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 4693
I20260924 21:20:01.429988 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 4331
I20260924 21:20:21.435513 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46259
--webserver_interface=127.20.115.254
--webserver_port=34215
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:46259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:21.515659 4854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:21.515852 4854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:21.515874 4854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:21.516952 4854 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260924 21:20:21.516983 4854 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260924 21:20:21.516995 4854 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260924 21:20:21.517007 4854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:21.517019 4854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:21.517032 4854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:21.517045 4854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:21.517056 4854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:21.518410 4854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:46259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:46259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=34215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:21.518591 4854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:21.518774 4854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:20:21.523758 4854 server_base.cc:1034] running on GCE node
W20260924 21:20:21.523806 4859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:21.523806 4862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:21.523885 4860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:21.524319 4854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:21.524600 4854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:21.525748 4854 hybrid_clock.cc:648] HybridClock initialized: now 1790284821525729 us; error 38 us; skew 500 ppm
Sep 24 21:20:21 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:21.528146 4854 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:20:21.528607 4854 webserver.cc:460] Webserver started at http://127.20.115.254:34215/ using document root <none> and password file <none>
I20260924 21:20:21.528896 4854 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:21.528966 4854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:21.530304 4854 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:21.530987 4869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:21.531167 4854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:21.531265 4854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "80cba6bf686646d28a3e4fdbb6607fa3"
format_stamp: "Formatted at 2026-09-24 21:19:39 on dist-test-slave-cxpd"
server_key: "8ccd1ae2bf31de35f6ce6f87c49e3518"
server_key_iv: "875b81a36dd3e73f6ece38605cb5c614"
server_key_version: "encryptionkey@0"
I20260924 21:20:21.531587 4854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:21.544319 4854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:21.546497 4854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:21.546653 4854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:21.550931 4854 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:46259
I20260924 21:20:21.551054 4921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:46259 every 8 connection(s)
I20260924 21:20:21.551355 4854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:21.552734 4922 sys_catalog.cc:263] Verifying existing consensus state
I20260924 21:20:21.553380 4922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Bootstrap starting.
I20260924 21:20:21.560750 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 4854
I20260924 21:20:21.561102 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:38773
--local_ip_for_outbound_sockets=127.20.115.193
--tserver_master_addrs=127.20.115.254:46259
--webserver_port=36697
--webserver_interface=127.20.115.193
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:20:21.562069 4922 log.cc:824] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:21.564275 4922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:20:21.564576 4922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Bootstrap complete.
I20260924 21:20:21.566823 4922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:20:21.567035 4922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80cba6bf686646d28a3e4fdbb6607fa3, State: Initialized, Role: FOLLOWER
I20260924 21:20:21.567201 4922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:20:21.567271 4922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:21.567332 4922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:21.567380 4922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 2 FOLLOWER]: Advancing to term 3
I20260924 21:20:21.568362 4922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:20:21.568450 4922 leader_election.cc:302] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 80cba6bf686646d28a3e4fdbb6607fa3; no voters:
I20260924 21:20:21.568660 4922 leader_election.cc:288] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260924 21:20:21.568686 4926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 3 FOLLOWER]: Leader election won for term 3
I20260924 21:20:21.568818 4926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [term 3 LEADER]: Becoming Leader. State: Replica: 80cba6bf686646d28a3e4fdbb6607fa3, State: Running, Role: LEADER
I20260924 21:20:21.568907 4926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } }
I20260924 21:20:21.568945 4922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:21.569190 4928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 80cba6bf686646d28a3e4fdbb6607fa3. Latest consensus state: current_term: 3 leader_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } } }
I20260924 21:20:21.569463 4928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:21.569387 4927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "80cba6bf686646d28a3e4fdbb6607fa3" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 46259 } } }
I20260924 21:20:21.569761 4927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:21.569772 4936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:21.570642 4936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:21.570746 4936 catalog_manager.cc:1259] Loaded cluster ID: 4804670773f74f50a65f73e792a44b67
I20260924 21:20:21.570784 4936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:21.571693 4936 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:21.571887 4936 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Loaded TSK: 0
I20260924 21:20:21.571946 4936 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 80cba6bf686646d28a3e4fdbb6607fa3: Loaded TSK: 1
F20260924 21:20:21.572182 4936 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f87064059ad google::LogMessage::Fail() at ??:0
@ 0x7f870640888a google::LogMessage::SendToLog() at ??:0
@ 0x7f87064054de google::LogMessage::Flush() at ??:0
@ 0x7f8706406a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8708b72104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f8708b6fd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f870e3631cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f870e36348e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f870e363594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f870e329c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f870e333ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f87082d30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f87082c8954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f870912c6db start_thread at ??:0
@ 0x7f8704f0361f clone at ??:0
W20260924 21:20:21.662989 4925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:21.663189 4925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:21.663213 4925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:21.664331 4925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:21.664371 4925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:21.664397 4925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:20:21.665880 4925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:38773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=36697
--tserver_master_addrs=127.20.115.254:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4925
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:21.666152 4925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:21.666399 4925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:21.671352 4964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:21.671365 4963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:21.671368 4966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:21.671789 4925 server_base.cc:1034] running on GCE node
I20260924 21:20:21.671993 4925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:21.672268 4925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:21.673418 4925 hybrid_clock.cc:648] HybridClock initialized: now 1790284821673401 us; error 34 us; skew 500 ppm
Sep 24 21:20:21 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:21.675828 4925 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:20:21.676299 4925 webserver.cc:460] Webserver started at http://127.20.115.193:36697/ using document root <none> and password file <none>
I20260924 21:20:21.676532 4925 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:21.676599 4925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:21.677977 4925 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:21.678694 4973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:21.678866 4925 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:21.678959 4925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "117a8e037e7740818096e4852d0242dd"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "533a9481ecbe7b9ab3af901a36ec879d"
server_key_iv: "a86592a9788fae9c55936538eb50d48d"
server_key_version: "encryptionkey@0"
I20260924 21:20:21.679307 4925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:21.694438 4925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:21.696559 4925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:21.696714 4925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:21.696962 4925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:21.697400 4980 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260924 21:20:21.699322 4925 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260924 21:20:21.699402 4925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260924 21:20:21.699450 4925 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260924 21:20:21.701743 4925 ts_tablet_manager.cc:610] Registered 1 tablets
I20260924 21:20:21.701824 4925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:21.701894 4980 tablet_bootstrap.cc:492] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Bootstrap starting.
I20260924 21:20:21.709262 4925 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:38773
I20260924 21:20:21.709405 5087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:38773 every 8 connection(s)
I20260924 21:20:21.709687 4925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:20:21.713171 4980 log.cc:824] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:21.715878 4980 tablet_bootstrap.cc:492] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260924 21:20:21.716203 4980 tablet_bootstrap.cc:492] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Bootstrap complete.
I20260924 21:20:21.716727 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 4925
I20260924 21:20:21.717028 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:40509
--local_ip_for_outbound_sockets=127.20.115.194
--tserver_master_addrs=127.20.115.254:46259
--webserver_port=38607
--webserver_interface=127.20.115.194
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:20:21.717410 4980 ts_tablet_manager.cc:1397] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.012s
I20260924 21:20:21.718772 4980 raft_consensus.cc:348] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:21.719105 4980 raft_consensus.cc:729] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 117a8e037e7740818096e4852d0242dd, State: Initialized, Role: FOLLOWER
I20260924 21:20:21.719255 4980 consensus_queue.cc:260] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:21.719528 4980 ts_tablet_manager.cc:1428] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260924 21:20:21.780103 5090 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:20:21.710394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:41691)
0924 21:20:21.710574 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:21.710618 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:21.711078 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:21.779871 (+ 68793us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
W20260924 21:20:21.780386 5088 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:46259 (0 consecutive failures): Network error: Failed to ping master at 127.20.115.254:46259: Client connection negotiation failed: client connection to 127.20.115.254:46259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260924 21:20:21.780723 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:21.780601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:57851)
0924 21:20:21.780657 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:21.780693 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:20:21.781093 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:21.781002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:43741)
0924 21:20:21.781047 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:21.781076 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:20:21.781219 5088 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260924 21:20:21.816447 5092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:21.816635 5092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:21.816658 5092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:21.817786 5092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:21.817824 5092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:21.817849 5092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:20:21.819190 5092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:40509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=38607
--tserver_master_addrs=127.20.115.254:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:21.819356 5092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:21.819515 5092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:21.824446 5099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:21.824466 5101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:21.824524 5092 server_base.cc:1034] running on GCE node
W20260924 21:20:21.824453 5098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:21.824888 5092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:21.825153 5092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:21.826339 5092 hybrid_clock.cc:648] HybridClock initialized: now 1790284821826322 us; error 25 us; skew 500 ppm
Sep 24 21:20:21 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:21.828879 5092 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:20:21.829262 5092 webserver.cc:460] Webserver started at http://127.20.115.194:38607/ using document root <none> and password file <none>
I20260924 21:20:21.829430 5092 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:21.829470 5092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:21.830729 5092 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:21.831287 5108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:21.831451 5092 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:21.831537 5092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "ce32df00534b4e29a67dd0c2a11b0374"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "a638a2670610d8c7cffdd647a1acddda"
server_key_iv: "ecc7610177ca6f074035603e529633e0"
server_key_version: "encryptionkey@0"
I20260924 21:20:21.831856 5092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:21.851331 5092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:21.853649 5092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:21.853808 5092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:21.854041 5092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:21.854358 5092 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:21.854415 5092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:21.854454 5092 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:21.854481 5092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:21.860222 5092 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:40509
I20260924 21:20:21.860270 5221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:40509 every 8 connection(s)
I20260924 21:20:21.860605 5092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:20:21.861562 5224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:21.861220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:39773)
0924 21:20:21.861359 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:21.861426 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
W20260924 21:20:21.861960 5222 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:46259 (0 consecutive failures): Network error: Failed to ping master at 127.20.115.254:46259: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
I20260924 21:20:21.862294 5224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:21.862203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:46873)
0924 21:20:21.862250 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:21.862276 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:20:21.862591 5224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:21.862517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:45327)
0924 21:20:21.862554 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:21.862575 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 21:20:21.862707 5222 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:20:21.863036 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 5092
I20260924 21:20:21.863292 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:45245
--local_ip_for_outbound_sockets=127.20.115.195
--tserver_master_addrs=127.20.115.254:46259
--webserver_port=44731
--webserver_interface=127.20.115.195
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42031
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:21.943491 5225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:21.943725 5225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:21.943763 5225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:21.944916 5225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:21.945003 5225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:21.945050 5225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:20:21.946480 5225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42031
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:45245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=44731
--tserver_master_addrs=127.20.115.254:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:21.946728 5225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:21.946946 5225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:21.951895 5231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:21.951906 5230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:21.951906 5233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:21.952168 5225 server_base.cc:1034] running on GCE node
I20260924 21:20:21.952369 5225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:21.952548 5225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:21.953696 5225 hybrid_clock.cc:648] HybridClock initialized: now 1790284821953684 us; error 29 us; skew 500 ppm
Sep 24 21:20:21 dist-test-slave-cxpd krb5kdc[3786](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:21.956215 5225 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:20:21.956614 5225 webserver.cc:460] Webserver started at http://127.20.115.195:44731/ using document root <none> and password file <none>
I20260924 21:20:21.956869 5225 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:21.956915 5225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:21.958549 5225 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:21.959177 5240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:21.959338 5225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:21.959467 5225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "c9c8c186b4f64b6686efa896ab96de40"
format_stamp: "Formatted at 2026-09-24 21:19:40 on dist-test-slave-cxpd"
server_key: "085058bd3a211f4e13096e29747d7b2e"
server_key_iv: "00ec6ae40b26b468e64c815bb8aa77c6"
server_key_version: "encryptionkey@0"
I20260924 21:20:21.959794 5225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:21.975840 5225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:21.978119 5225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:21.978277 5225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:21.978499 5225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:21.978783 5225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:21.978837 5225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:21.978876 5225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:21.978904 5225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:21.985163 5225 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:45245
I20260924 21:20:21.985220 5353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:45245 every 8 connection(s)
I20260924 21:20:21.985529 5225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:20:21.986449 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:21.986106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:37561)
0924 21:20:21.986256 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:21.986319 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":80,"threads_started":1}
W20260924 21:20:21.986806 5354 heartbeater.cc:644] Failed to heartbeat to 127.20.115.254:46259 (0 consecutive failures): Network error: Failed to ping master at 127.20.115.254:46259: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
I20260924 21:20:21.987073 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:21.986981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:54211)
0924 21:20:21.987027 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:21.987056 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 21:20:21.987349 5356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:21.987266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:60799)
0924 21:20:21.987306 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:21.987327 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260924 21:20:21.987444 5354 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260924 21:20:21.987993 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 5225
I20260924 21:20:22.781874 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:22.781620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:43567)
0924 21:20:22.781787 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:22.781825 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":58,"threads_started":1}
I20260924 21:20:22.863448 5359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:22.863197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:51273)
0924 21:20:22.863333 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:22.863391 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260924 21:20:22.987950 5360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:22.987737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:45759)
0924 21:20:22.987868 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:22.987903 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":53,"threads_started":1}
I20260924 21:20:23.097183 5361 raft_consensus.cc:482] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:23.097360 5361 raft_consensus.cc:504] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:23.097703 5361 leader_election.cc:288] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245), ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509)
Sep 24 21:20:23 dist-test-slave-cxpd krb5kdc[3786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.194@KRBTEST.COM
Sep 24 21:20:23 dist-test-slave-cxpd krb5kdc[3786](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284821, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.195@KRBTEST.COM
I20260924 21:20:23.108160 5363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:23.098138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:39761 (local address 127.20.115.195:45245)
0924 21:20:23.098280 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:23.098290 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:23.098315 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:20:23.098756 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:23.098758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:23.098799 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:23.098958 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:23.099718 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:23.100717 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:23.101728 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:23.102139 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:23.102270 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:23.104817 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:23.104820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:23.104820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:23.104832 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:23.106442 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:23.106924 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:23.106927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:23.106928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:23.107020 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:23.107302 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:23.107304 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:23.107304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:23.107537 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:23.107633 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:23.107911 (+ 278us) server_negotiation.cc:299] Negotiation successful
0924 21:20:23.108040 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260924 21:20:23.108145 5358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:23.097944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:45245 (local address 127.20.115.193:39761)
0924 21:20:23.098008 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:23.098021 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:23.098097 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:23.098996 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:23.098999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:23.099386 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:23.099571 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:23.099578 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:23.100853 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:23.100854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:23.101624 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:23.101626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:23.102238 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:23.102239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:23.102347 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:23.102987 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:23.102992 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:23.104683 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:23.106549 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:23.106550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:23.106550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:23.106836 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:23.107091 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:23.107091 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:23.107092 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:23.107245 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:23.107619 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:23.107620 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:23.107749 (+ 129us) client_negotiation.cc:769] Sending connection context
0924 21:20:23.107856 (+ 107us) client_negotiation.cc:240] Negotiation successful
0924 21:20:23.108053 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":64}
I20260924 21:20:23.108180 5364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:23.098181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:58321 (local address 127.20.115.194:40509)
0924 21:20:23.098340 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:23.098359 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:23.098383 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:20:23.098803 (+ 420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:23.098805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:23.098841 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:23.098986 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:23.099696 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:23.100708 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:23.101722 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:23.102140 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:23.102262 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:23.104399 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:23.104402 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:23.104402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:23.104412 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:23.106428 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:23.106932 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:23.106935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:23.106936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:23.107027 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:23.107289 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:23.107290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:23.107290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:23.107530 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:23.107630 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:23.107906 (+ 276us) server_negotiation.cc:299] Negotiation successful
0924 21:20:23.108036 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":69,"threads_started":1}
I20260924 21:20:23.108264 5362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:23.098072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:40509 (local address 127.20.115.193:58321)
0924 21:20:23.098165 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:23.098176 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:23.098233 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:23.098997 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:23.099000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:23.099385 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:23.099572 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:23.099578 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:23.100860 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:23.100861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:23.101623 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:23.101625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:23.102232 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:23.102233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:23.102353 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:23.102815 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:23.102830 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:23.104288 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:23.106512 (+ 2224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:23.106519 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:23.106521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:23.106836 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:23.107116 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:23.107117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:23.107117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:23.107208 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:23.107609 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:23.107616 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:23.107747 (+ 131us) client_negotiation.cc:769] Sending connection context
0924 21:20:23.107855 (+ 108us) client_negotiation.cc:240] Negotiation successful
0924 21:20:23.108136 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":145,"thread_start_us":47,"threads_started":1}
W20260924 21:20:23.108980 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
W20260924 21:20:23.109522 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
I20260924 21:20:23.109609 4977 leader_election.cc:302] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117a8e037e7740818096e4852d0242dd; no voters: c9c8c186b4f64b6686efa896ab96de40, ce32df00534b4e29a67dd0c2a11b0374
I20260924 21:20:23.109786 5361 raft_consensus.cc:2726] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:20:23.782675 5365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:23.782389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:33325)
0924 21:20:23.782522 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:23.782609 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260924 21:20:23.864140 5366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:23.863908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:37047)
0924 21:20:23.864031 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:23.864095 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260924 21:20:23.988646 5367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:23.988439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:49483)
0924 21:20:23.988571 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:23.988605 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260924 21:20:24.689137 5368 raft_consensus.cc:482] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:24.689236 5368 raft_consensus.cc:504] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:24.689383 5368 leader_election.cc:288] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245), ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509)
W20260924 21:20:24.689800 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
W20260924 21:20:24.689869 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
I20260924 21:20:24.689885 4977 leader_election.cc:302] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117a8e037e7740818096e4852d0242dd; no voters: c9c8c186b4f64b6686efa896ab96de40, ce32df00534b4e29a67dd0c2a11b0374
I20260924 21:20:24.689989 5368 raft_consensus.cc:2726] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:20:24.783368 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:24.783141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:50795)
0924 21:20:24.783282 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:24.783323 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
I20260924 21:20:24.864874 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:24.864606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:39191)
0924 21:20:24.864752 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:24.864826 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":79,"threads_started":1}
I20260924 21:20:24.989419 5371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:24.989204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:40767)
0924 21:20:24.989335 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:24.989376 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":60,"threads_started":1}
I20260924 21:20:25.783979 5372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:25.783776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:52591)
0924 21:20:25.783901 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:25.783935 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":53,"threads_started":1}
I20260924 21:20:25.865615 5373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:25.865349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:41605)
0924 21:20:25.865525 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:25.865560 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":70,"threads_started":1}
I20260924 21:20:25.990156 5374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:25.989900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:42429)
0924 21:20:25.990075 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:25.990114 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":64,"threads_started":1}
I20260924 21:20:26.784724 5375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:26.784519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:37181)
0924 21:20:26.784643 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:26.784679 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":60,"threads_started":1}
I20260924 21:20:26.866272 5376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:26.866081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:38869)
0924 21:20:26.866203 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:26.866234 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":57,"threads_started":1}
I20260924 21:20:26.990906 5377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:26.990676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:43107)
0924 21:20:26.990827 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:26.990863 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":96,"threads_started":1}
I20260924 21:20:27.785552 5378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:27.785339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:40977)
0924 21:20:27.785462 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:27.785502 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
I20260924 21:20:27.866919 5379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:27.866703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:41821)
0924 21:20:27.866835 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:27.866870 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260924 21:20:27.925225 5380 raft_consensus.cc:482] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:27.925319 5380 raft_consensus.cc:504] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:27.925465 5380 leader_election.cc:288] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245), ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509)
W20260924 21:20:27.925923 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
W20260924 21:20:27.925979 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
I20260924 21:20:27.925993 4977 leader_election.cc:302] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117a8e037e7740818096e4852d0242dd; no voters: c9c8c186b4f64b6686efa896ab96de40, ce32df00534b4e29a67dd0c2a11b0374
I20260924 21:20:27.926093 5380 raft_consensus.cc:2726] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:20:27.991559 5381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:27.991325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:53369)
0924 21:20:27.991483 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:27.991518 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":71,"threads_started":1}
I20260924 21:20:28.786393 5382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:28.786111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:44901)
0924 21:20:28.786308 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:28.786347 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":58,"threads_started":1}
I20260924 21:20:28.867600 5383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:28.867396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:50351)
0924 21:20:28.867516 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:28.867557 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260924 21:20:28.992290 5384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:28.991999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:42199)
0924 21:20:28.992215 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:28.992250 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":58,"threads_started":1}
I20260924 21:20:29.787125 5385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:29.786913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:57049)
0924 21:20:29.787046 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:29.787081 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260924 21:20:29.868233 5386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:29.868040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:45853)
0924 21:20:29.868160 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:29.868194 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
I20260924 21:20:29.992935 5387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:29.992710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:32953)
0924 21:20:29.992863 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:29.992895 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":57,"threads_started":1}
I20260924 21:20:30.517655 5388 raft_consensus.cc:482] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:30.517769 5388 raft_consensus.cc:504] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:30.517956 5388 leader_election.cc:288] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245), ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509)
W20260924 21:20:30.518391 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
W20260924 21:20:30.518467 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
I20260924 21:20:30.518486 4977 leader_election.cc:302] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117a8e037e7740818096e4852d0242dd; no voters: c9c8c186b4f64b6686efa896ab96de40, ce32df00534b4e29a67dd0c2a11b0374
I20260924 21:20:30.518587 5388 raft_consensus.cc:2726] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:20:30.787853 5389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:30.787634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:57573)
0924 21:20:30.787770 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:30.787808 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260924 21:20:30.869020 5390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:30.868800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:57235)
0924 21:20:30.868918 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:30.868958 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":54,"threads_started":1}
I20260924 21:20:30.993664 5391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:30.993411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:53623)
0924 21:20:30.993565 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:30.993605 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":61,"threads_started":1}
I20260924 21:20:31.788658 5392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:31.788368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:35295)
0924 21:20:31.788520 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:31.788590 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":73,"threads_started":1}
I20260924 21:20:31.869841 5393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:31.869612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:57489)
0924 21:20:31.869759 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:31.869797 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":59,"threads_started":1}
I20260924 21:20:31.994335 5394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:31.994130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:55087)
0924 21:20:31.994261 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:31.994296 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260924 21:20:32.789434 5395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:32.789200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:46885)
0924 21:20:32.789345 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:32.789388 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260924 21:20:32.870748 5396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:32.870424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:33157)
0924 21:20:32.870652 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:32.870695 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":181,"threads_started":1}
I20260924 21:20:32.995117 5397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:32.994888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:43509)
0924 21:20:32.995037 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:32.995074 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
I20260924 21:20:33.578559 5398 raft_consensus.cc:482] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:33.578652 5398 raft_consensus.cc:504] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9c8c186b4f64b6686efa896ab96de40" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 45245 } } peers { permanent_uuid: "ce32df00534b4e29a67dd0c2a11b0374" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 40509 } } peers { permanent_uuid: "117a8e037e7740818096e4852d0242dd" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 38773 } }
I20260924 21:20:33.578773 5398 leader_election.cc:288] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245), ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509)
W20260924 21:20:33.579226 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c9c8c186b4f64b6686efa896ab96de40 (127.20.115.195:45245): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
W20260924 21:20:33.579437 4977 leader_election.cc:341] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ce32df00534b4e29a67dd0c2a11b0374 (127.20.115.194:40509): Not found: Tablet not found: 0ec02675c74a4e9ab0fe7ffebbd8c22e
I20260924 21:20:33.579506 4977 leader_election.cc:302] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 117a8e037e7740818096e4852d0242dd; no voters: c9c8c186b4f64b6686efa896ab96de40, ce32df00534b4e29a67dd0c2a11b0374
I20260924 21:20:33.579592 5398 raft_consensus.cc:2726] T 0ec02675c74a4e9ab0fe7ffebbd8c22e P 117a8e037e7740818096e4852d0242dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260924 21:20:33.790162 5399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:33.789925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:42597)
0924 21:20:33.790080 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:33.790117 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":75,"threads_started":1}
I20260924 21:20:33.871464 5400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:33.871239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:39797)
0924 21:20:33.871376 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:33.871417 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":59,"threads_started":1}
I20260924 21:20:33.995827 5401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:33.995629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:40865)
0924 21:20:33.995747 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:33.995783 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260924 21:20:34.790841 5402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:34.790628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:60765)
0924 21:20:34.790762 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:34.790795 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260924 21:20:34.872181 5403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:34.871965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:35289)
0924 21:20:34.872102 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:34.872139 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":56,"threads_started":1}
I20260924 21:20:34.996520 5404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:34.996290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:48557)
0924 21:20:34.996430 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:34.996464 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260924 21:20:35.791673 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:35.791428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:47255)
0924 21:20:35.791585 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:35.791626 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":72,"threads_started":1}
I20260924 21:20:35.872921 5406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:35.872650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:44213)
0924 21:20:35.872832 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:35.872876 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260924 21:20:35.997241 5407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:35.997021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:58357)
0924 21:20:35.997163 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:35.997197 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":59,"threads_started":1}
I20260924 21:20:36.792703 5408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:36.792475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.193:58797)
0924 21:20:36.792624 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:36.792658 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":62,"threads_started":1}
I20260924 21:20:36.873600 5409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:36.873382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.194:33111)
0924 21:20:36.873523 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:36.873558 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
I20260924 21:20:36.989912 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 4925
I20260924 21:20:36.994596 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 5092
I20260924 21:20:36.997977 5410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:36.997779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:46259 (local address 127.20.115.195:54375)
0924 21:20:36.997904 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:36.997936 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.254:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":73,"threads_started":1}
I20260924 21:20:36.998973 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 5225
2026-09-24T21:20:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58411 ms)
[----------] 30 tests from SecurityITest (230134 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:37 dist-test-slave-cxpd krb5kdc[5413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:37 dist-test-slave-cxpd krb5kdc[5413](info): set up 2 sockets
Sep 24 21:20:37 dist-test-slave-cxpd krb5kdc[5413](info): commencing operation
krb5kdc: starting...
W20260924 21:20:38.032402 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:38 dist-test-slave-cxpd krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:20:38.152318 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:36535
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:36535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:38.232229 5426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:38.232434 5426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:38.232456 5426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:38.233592 5426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:38.233634 5426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:38.233649 5426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:38.233662 5426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:38.233675 5426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:38.235106 5426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:36535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:36535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:38.235294 5426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:38.235468 5426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:38.240731 5432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:38.240746 5434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:38.240849 5426 server_base.cc:1034] running on GCE node
W20260924 21:20:38.240743 5431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:38.241214 5426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:38.241550 5426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:38.242713 5426 hybrid_clock.cc:648] HybridClock initialized: now 1790284838242703 us; error 38 us; skew 500 ppm
Sep 24 21:20:38 dist-test-slave-cxpd krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:38.245312 5426 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:20:38.245713 5426 webserver.cc:460] Webserver started at http://127.20.115.254:39709/ using document root <none> and password file <none>
I20260924 21:20:38.245911 5426 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:38.245951 5426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:38.246038 5426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:38.247126 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "4759612137fd4281a64f7f3b84eb7122"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "17d1587e3ec52cb39620413f4ce75dfa"
server_key_iv: "3420f2f10dd912214b062fb1f6d9a5b5"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.247423 5426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "4759612137fd4281a64f7f3b84eb7122"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "17d1587e3ec52cb39620413f4ce75dfa"
server_key_iv: "3420f2f10dd912214b062fb1f6d9a5b5"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.248726 5426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:20:38.249737 5441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.249944 5426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:20:38.250057 5426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "4759612137fd4281a64f7f3b84eb7122"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "17d1587e3ec52cb39620413f4ce75dfa"
server_key_iv: "3420f2f10dd912214b062fb1f6d9a5b5"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.250168 5426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:38.261210 5426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:38.263029 5426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:38.263203 5426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:38.267158 5426 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:36535
I20260924 21:20:38.267263 5493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:36535 every 8 connection(s)
I20260924 21:20:38.267587 5426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:38.268533 5494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:38.270853 5494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122: Bootstrap starting.
I20260924 21:20:38.271535 5494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:38.271883 5494 log.cc:824] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:38.272720 5494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122: No bootstrap required, opened a new log
I20260924 21:20:38.274933 5494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4759612137fd4281a64f7f3b84eb7122" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36535 } }
I20260924 21:20:38.275098 5494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:38.275125 5494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4759612137fd4281a64f7f3b84eb7122, State: Initialized, Role: FOLLOWER
I20260924 21:20:38.275288 5494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4759612137fd4281a64f7f3b84eb7122" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36535 } }
I20260924 21:20:38.275363 5494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:38.275386 5494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:38.275434 5494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:38.275962 5494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4759612137fd4281a64f7f3b84eb7122" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36535 } }
I20260924 21:20:38.276047 5494 leader_election.cc:302] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4759612137fd4281a64f7f3b84eb7122; no voters:
I20260924 21:20:38.276232 5494 leader_election.cc:288] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:38.276299 5497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:38.276458 5497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [term 1 LEADER]: Becoming Leader. State: Replica: 4759612137fd4281a64f7f3b84eb7122, State: Running, Role: LEADER
I20260924 21:20:38.276484 5494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:38.276564 5497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4759612137fd4281a64f7f3b84eb7122" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36535 } }
I20260924 21:20:38.277158 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 5426
I20260924 21:20:38.277285 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:38.277452 20943 external_mini_cluster.cc:1428] Setting key 3dfb725414ef0699bc0a6b1566cd77d0
I20260924 21:20:38.277783 5499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4759612137fd4281a64f7f3b84eb7122. Latest consensus state: current_term: 1 leader_uuid: "4759612137fd4281a64f7f3b84eb7122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4759612137fd4281a64f7f3b84eb7122" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36535 } } }
I20260924 21:20:38.277873 5498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4759612137fd4281a64f7f3b84eb7122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4759612137fd4281a64f7f3b84eb7122" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 36535 } } }
I20260924 21:20:38.278056 5498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:38.277882 5499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:38.278568 5509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:38.279318 5509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:38.280921 5509 catalog_manager.cc:1347] Generated new cluster ID: 4849ddaee3fd4645b1829ef33b00c219
I20260924 21:20:38.280982 5509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:20:38 dist-test-slave-cxpd krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284838, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:38.285825 5509 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:38.286448 5509 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:38.289767 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.278132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32920 (local address 127.20.115.254:36535)
0924 21:20:38.278360 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.278370 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.278444 (+ 74us) server_negotiation.cc:407] Connection header received
0924 21:20:38.278740 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.278747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.278816 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.278943 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:38.279810 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.281369 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.282441 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.283097 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.283209 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.286173 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:38.286177 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:38.286177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:38.286188 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:38.288109 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.288514 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.288517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.288517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.288616 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.288850 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.288851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.288851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.289116 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:38.289265 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.289386 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.289536 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":85,"threads_started":1}
I20260924 21:20:38.293437 5509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122: Generated new TSK 0
I20260924 21:20:38.293610 5509 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:20:38.323668 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36535
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:38.404438 5520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:38.404641 5520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:38.404668 5520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:38.405824 5520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:38.405869 5520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:38.405895 5520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:20:38.407336 5520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:38.407564 5520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:38.407817 5520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:20:38.412832 5520 server_base.cc:1034] running on GCE node
W20260924 21:20:38.412801 5525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:38.412796 5528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:38.412801 5526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:38.413341 5520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:38.413584 5520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:38.414729 5520 hybrid_clock.cc:648] HybridClock initialized: now 1790284838414721 us; error 31 us; skew 500 ppm
Sep 24 21:20:38 dist-test-slave-cxpd krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:38.417342 5520 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:20:38.417840 5520 webserver.cc:460] Webserver started at http://127.20.115.193:39357/ using document root <none> and password file <none>
I20260924 21:20:38.418079 5520 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:38.418146 5520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:38.418283 5520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:38.419328 5520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "576936364b98444e9b13805068b7f8c7"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "38a6fd9103975c1514105c76d1ac6dc0"
server_key_iv: "07129755c67c6ea950f167f709aec6b2"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.419688 5520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "576936364b98444e9b13805068b7f8c7"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "38a6fd9103975c1514105c76d1ac6dc0"
server_key_iv: "07129755c67c6ea950f167f709aec6b2"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.421166 5520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:20:38.421896 5535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.422119 5520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:38.422226 5520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "576936364b98444e9b13805068b7f8c7"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "38a6fd9103975c1514105c76d1ac6dc0"
server_key_iv: "07129755c67c6ea950f167f709aec6b2"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.422314 5520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:38.431180 5520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:38.433075 5520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:38.433255 5520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:38.433457 5520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:38.433737 5520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:38.433790 5520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.433830 5520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:38.433857 5520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.439895 5520 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:36945
I20260924 21:20:38.439939 5648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:36945 every 8 connection(s)
I20260924 21:20:38.440251 5520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
Sep 24 21:20:38 dist-test-slave-cxpd krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:38.448225 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 5520
I20260924 21:20:38.448350 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:20:38.448544 20943 external_mini_cluster.cc:1428] Setting key 128cd7bb29bd763f3e3a765cfb8647ea
I20260924 21:20:38.449651 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.440954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:48939 (local address 127.20.115.254:36535)
0924 21:20:38.441043 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.441045 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.441281 (+ 236us) server_negotiation.cc:407] Connection header received
0924 21:20:38.441743 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.441744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.441759 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.441815 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:38.442730 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.443251 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.444221 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.444425 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.444488 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.446714 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:38.446715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:38.446715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:38.446722 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:38.447926 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.448365 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.448367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.448367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.448459 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.448848 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.448849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.448849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.448986 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:38.449044 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.449418 (+ 374us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.449516 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 21:20:38.449676 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.440939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36535 (local address 127.20.115.193:48939)
0924 21:20:38.441157 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.441194 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:38.441660 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:38.441865 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:38.441868 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:38.442427 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:38.442620 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.442624 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.443345 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.443345 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.444128 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.444130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.444537 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.444538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.444659 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.445216 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:38.445224 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:38.446624 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:38.448001 (+ 1377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:38.448008 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:38.448009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:38.448249 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:38.448537 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:38.448538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:38.448538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:38.448715 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:38.449058 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:38.449064 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:38.449258 (+ 194us) client_negotiation.cc:769] Sending connection context
0924 21:20:38.449378 (+ 120us) client_negotiation.cc:240] Negotiation successful
0924 21:20:38.449507 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":137,"threads_started":1}
I20260924 21:20:38.450238 5649 heartbeater.cc:347] Connected to a master server at 127.20.115.254:36535
I20260924 21:20:38.450353 5649 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:38.450528 5649 heartbeater.cc:511] Master 127.20.115.254:36535 requested a full tablet report, sending...
I20260924 21:20:38.451040 5458 ts_manager.cc:194] Registered new tserver with Master: 576936364b98444e9b13805068b7f8c7 (127.20.115.193:36945)
I20260924 21:20:38.451663 5458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:48939
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:20:38.474092 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36535
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:38.552960 5654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:38.553217 5654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:38.553246 5654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:38.554368 5654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:38.554409 5654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:38.554462 5654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:20:38.555898 5654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:38.556123 5654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:38.556336 5654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:38.561076 5662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:38.561148 5654 server_base.cc:1034] running on GCE node
W20260924 21:20:38.561107 5659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:38.561079 5660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:38.561486 5654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:38.561678 5654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:38.562814 5654 hybrid_clock.cc:648] HybridClock initialized: now 1790284838562800 us; error 28 us; skew 500 ppm
Sep 24 21:20:38 dist-test-slave-cxpd krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:38.565338 5654 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:20:38.565810 5654 webserver.cc:460] Webserver started at http://127.20.115.194:46415/ using document root <none> and password file <none>
I20260924 21:20:38.566083 5654 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:38.566155 5654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:38.566308 5654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:38.567382 5654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "051824f623a44b27832e5bab0f363175"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "5cad9d0709373819d33f6f6092ff29a7"
server_key_iv: "b609146f5c4d536e0ce7d8c475605572"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.567750 5654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "051824f623a44b27832e5bab0f363175"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "5cad9d0709373819d33f6f6092ff29a7"
server_key_iv: "b609146f5c4d536e0ce7d8c475605572"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.569319 5654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:20:38.570099 5669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.570375 5654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:38.570488 5654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "051824f623a44b27832e5bab0f363175"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "5cad9d0709373819d33f6f6092ff29a7"
server_key_iv: "b609146f5c4d536e0ce7d8c475605572"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.570564 5654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:38.585327 5654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:38.587327 5654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:38.587491 5654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:38.587723 5654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:38.588033 5654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:38.588088 5654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.588130 5654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:38.588158 5654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.593861 5654 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:41775
I20260924 21:20:38.593910 5782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:41775 every 8 connection(s)
I20260924 21:20:38.594210 5654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:20:38.599148 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 5654
I20260924 21:20:38.599291 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:20:38.599506 20943 external_mini_cluster.cc:1428] Setting key 7687b72d231d1233f915454ab8d5038d
Sep 24 21:20:38 dist-test-slave-cxpd krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:38.603832 5785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.594835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36535 (local address 127.20.115.194:43009)
0924 21:20:38.594973 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.595027 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:38.595551 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:38.595815 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:38.595819 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:38.596387 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:38.596575 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.596580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.597339 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.597340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.598149 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.598151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.598544 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.598545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.598673 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.599235 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:38.599243 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:38.600713 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:38.602397 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:38.602403 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:38.602406 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:38.602716 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:38.602943 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:38.602944 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:38.602944 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:38.603041 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:38.603330 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:38.603335 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:38.603453 (+ 118us) client_negotiation.cc:769] Sending connection context
0924 21:20:38.603543 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 21:20:38.603666 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":67,"threads_started":1}
I20260924 21:20:38.603826 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.594870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:43009 (local address 127.20.115.254:36535)
0924 21:20:38.594914 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.594917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.595120 (+ 203us) server_negotiation.cc:407] Connection header received
0924 21:20:38.595672 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.595674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.595687 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.595742 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:38.596680 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.597244 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.598266 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.598461 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.598537 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.600831 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:38.600833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:38.600833 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:38.600842 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:38.602314 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.602815 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.602816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.602817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.602876 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.603105 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.603106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.603106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.603258 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:38.603326 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.603598 (+ 272us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.603688 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:20:38.604390 5783 heartbeater.cc:347] Connected to a master server at 127.20.115.254:36535
I20260924 21:20:38.604509 5783 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:38.604696 5783 heartbeater.cc:511] Master 127.20.115.254:36535 requested a full tablet report, sending...
I20260924 21:20:38.605178 5458 ts_manager.cc:194] Registered new tserver with Master: 051824f623a44b27832e5bab0f363175 (127.20.115.194:41775)
I20260924 21:20:38.605554 5458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:43009
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:20:38.628201 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36535
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:37329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:38.721125 5788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:38.721354 5788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:38.721391 5788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:38.722507 5788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:38.722579 5788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:38.722633 5788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:20:38.724066 5788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:37329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:36535
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:38.724326 5788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:38.724547 5788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:38.729624 5796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:38.729633 5793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:38.729660 5794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:38.729889 5788 server_base.cc:1034] running on GCE node
I20260924 21:20:38.730095 5788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:38.730351 5788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:38.731505 5788 hybrid_clock.cc:648] HybridClock initialized: now 1790284838731485 us; error 34 us; skew 500 ppm
Sep 24 21:20:38 dist-test-slave-cxpd krb5kdc[5413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:38.733958 5788 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:20:38.734397 5788 webserver.cc:460] Webserver started at http://127.20.115.195:36531/ using document root <none> and password file <none>
I20260924 21:20:38.734566 5788 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:38.734604 5788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:38.734690 5788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:38.735627 5788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "04de93c244694b6d853d3554d552b893"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "f0896fcd8bcbe58ea3017a50aaa9205a"
server_key_iv: "c83fa401f5ecc7fa9fb4b16a46f1d501"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.735912 5788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "04de93c244694b6d853d3554d552b893"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "f0896fcd8bcbe58ea3017a50aaa9205a"
server_key_iv: "c83fa401f5ecc7fa9fb4b16a46f1d501"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.737185 5788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:20:38.737849 5803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.738065 5788 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:20:38.738147 5788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "04de93c244694b6d853d3554d552b893"
format_stamp: "Formatted at 2026-09-24 21:20:38 on dist-test-slave-cxpd"
server_key: "f0896fcd8bcbe58ea3017a50aaa9205a"
server_key_iv: "c83fa401f5ecc7fa9fb4b16a46f1d501"
server_key_version: "encryptionkey@0"
I20260924 21:20:38.738197 5788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:38.750660 5788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:38.752452 5788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:38.752583 5788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:38.752761 5788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:38.753126 5788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:38.753159 5788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.753180 5788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:38.753193 5788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:38.758671 5788 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:33105
I20260924 21:20:38.758745 5916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:33105 every 8 connection(s)
I20260924 21:20:38.759025 5788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:20:38.763237 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 5788
I20260924 21:20:38.763355 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:20:38.763538 20943 external_mini_cluster.cc:1428] Setting key daa345e7a1e1cfa4892b507a80830a70
Sep 24 21:20:38 dist-test-slave-cxpd krb5kdc[5413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284838, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:38.767729 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.759673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:55461 (local address 127.20.115.254:36535)
0924 21:20:38.759740 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.759742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.759931 (+ 189us) server_negotiation.cc:407] Connection header received
0924 21:20:38.760439 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.760440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.760452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.760514 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:38.761498 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.762023 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.762992 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.763206 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.763290 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.765484 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:38.765485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:38.765486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:38.765492 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:38.766613 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.766992 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.766992 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.766993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.767040 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.767255 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.767256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.767256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.767349 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:38.767397 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.767610 (+ 213us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.767669 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:20:38.767925 5919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.759656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:36535 (local address 127.20.115.195:55461)
0924 21:20:38.759822 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.759854 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:38.760336 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:38.760584 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:38.760588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:38.761229 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:38.761400 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.761404 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.762117 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.762119 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.762891 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.762893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.763295 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.763296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.763435 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.763952 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:38.763961 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:38.765390 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:38.766727 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:38.766732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:38.766734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:38.766928 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:38.767107 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:38.767108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:38.767108 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:38.767202 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:38.767403 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:38.767406 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:38.767505 (+ 99us) client_negotiation.cc:769] Sending connection context
0924 21:20:38.767585 (+ 80us) client_negotiation.cc:240] Negotiation successful
0924 21:20:38.767697 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260924 21:20:38.768373 5917 heartbeater.cc:347] Connected to a master server at 127.20.115.254:36535
I20260924 21:20:38.768443 5917 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:38.768625 5917 heartbeater.cc:511] Master 127.20.115.254:36535 requested a full tablet report, sending...
I20260924 21:20:38.768999 5458 ts_manager.cc:194] Registered new tserver with Master: 04de93c244694b6d853d3554d552b893 (127.20.115.195:33105)
I20260924 21:20:38.769335 5458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:55461
I20260924 21:20:38.775192 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:20:38.782568 5511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.776505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32934 (local address 127.20.115.254:36535)
0924 21:20:38.776561 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.776562 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.776709 (+ 147us) server_negotiation.cc:407] Connection header received
0924 21:20:38.776800 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.776801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.776811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.776865 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:38.777521 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.778054 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.778871 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.779073 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.779163 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.780217 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:38.780218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:38.780219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:38.780225 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:38.781446 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.781796 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.781797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.781798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.781854 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:38.782047 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:38.782048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:38.782048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:38.782184 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:38.782286 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.782405 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.782475 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 21:20:38.784042 5458 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32934:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:20:38.785671 5458 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:20:38.791886 5925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.787617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:33105 (local address 127.0.0.1:36764)
0924 21:20:38.788131 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.788141 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:38.788210 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:38.788386 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:38.788389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:38.788391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:38.788466 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.788468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.789771 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.789772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.790735 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.790737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.791585 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.791586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.791665 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.791678 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:20:38.791750 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:20:38.791799 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":82,"threads_started":1}
I20260924 21:20:38.791896 5927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.787711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36764 (local address 127.20.115.195:33105)
0924 21:20:38.787887 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.787891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.788211 (+ 320us) server_negotiation.cc:407] Connection header received
0924 21:20:38.788262 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.788264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.788276 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.788357 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:38.788557 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.789659 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.790816 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.791489 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.791568 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.791624 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.791774 (+ 150us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.791815 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":86,"threads_started":1}
I20260924 21:20:38.792083 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.787972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41580 (local address 127.20.115.194:41775)
0924 21:20:38.788117 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.788122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.788328 (+ 206us) server_negotiation.cc:407] Connection header received
0924 21:20:38.788395 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.788398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.788414 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.788489 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:38.788711 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.789423 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.790893 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.791567 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.791626 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.791669 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.791935 (+ 266us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.791983 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":75,"threads_started":1}
I20260924 21:20:38.792256 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.787615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:36945 (local address 127.0.0.1:55948)
0924 21:20:38.787939 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.787975 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:38.788029 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:38.788672 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:38.788672 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:38.788674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:38.788715 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.788717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.789956 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.789957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.790988 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.790990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.791747 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.791748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.791801 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.791804 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:38.791850 (+ 46us) client_negotiation.cc:240] Negotiation successful
0924 21:20:38.791884 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"mutex_wait_us":1,"thread_start_us":80,"threads_started":1}
I20260924 21:20:38.792357 5926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.787838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:41775 (local address 127.0.0.1:41580)
0924 21:20:38.788267 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.788274 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:38.788326 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:38.788582 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:38.788583 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:38.788585 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:38.788628 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.788629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.789534 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.789535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.790814 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.790817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.791702 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.791703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.791791 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.791795 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:38.791845 (+ 50us) client_negotiation.cc:240] Negotiation successful
0924 21:20:38.791889 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":53,"threads_started":1}
I20260924 21:20:38.792464 5928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.787820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55948 (local address 127.20.115.193:36945)
0924 21:20:38.788458 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.788463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.788471 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:20:38.788509 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.788511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.788524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.788594 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:38.788800 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.789855 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.791071 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.791586 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.791658 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.791712 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.791925 (+ 213us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.791973 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":68,"threads_started":1}
I20260924 21:20:38.792685 5851 tablet_service.cc:1467] Processing CreateTablet for tablet 2f3a32281dc045ba8f8624f4b19990ae (DEFAULT_TABLE table=test-table [id=3942270b6f1d4db88a72b7824b07a208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:38.793054 5851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f3a32281dc045ba8f8624f4b19990ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:38.792999 5717 tablet_service.cc:1467] Processing CreateTablet for tablet 2f3a32281dc045ba8f8624f4b19990ae (DEFAULT_TABLE table=test-table [id=3942270b6f1d4db88a72b7824b07a208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:38.793084 5583 tablet_service.cc:1467] Processing CreateTablet for tablet 2f3a32281dc045ba8f8624f4b19990ae (DEFAULT_TABLE table=test-table [id=3942270b6f1d4db88a72b7824b07a208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:38.793295 5717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f3a32281dc045ba8f8624f4b19990ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:38.793383 5583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f3a32281dc045ba8f8624f4b19990ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:38.795818 5931 tablet_bootstrap.cc:492] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175: Bootstrap starting.
I20260924 21:20:38.796322 5932 tablet_bootstrap.cc:492] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: Bootstrap starting.
I20260924 21:20:38.796569 5933 tablet_bootstrap.cc:492] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7: Bootstrap starting.
I20260924 21:20:38.796595 5931 tablet_bootstrap.cc:654] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:38.797035 5932 tablet_bootstrap.cc:654] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:38.797180 5933 tablet_bootstrap.cc:654] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:38.797197 5931 log.cc:824] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:38.797549 5932 log.cc:824] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:38.797793 5933 log.cc:824] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:38.798054 5931 tablet_bootstrap.cc:492] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175: No bootstrap required, opened a new log
I20260924 21:20:38.798130 5931 ts_tablet_manager.cc:1397] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260924 21:20:38.798276 5932 tablet_bootstrap.cc:492] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: No bootstrap required, opened a new log
I20260924 21:20:38.798339 5932 ts_tablet_manager.cc:1397] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:38.798506 5933 tablet_bootstrap.cc:492] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7: No bootstrap required, opened a new log
I20260924 21:20:38.798559 5933 ts_tablet_manager.cc:1397] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260924 21:20:38.799612 5931 raft_consensus.cc:348] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } }
I20260924 21:20:38.799741 5931 raft_consensus.cc:374] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:38.799760 5931 raft_consensus.cc:729] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051824f623a44b27832e5bab0f363175, State: Initialized, Role: FOLLOWER
I20260924 21:20:38.799876 5931 consensus_queue.cc:260] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } }
I20260924 21:20:38.800122 5931 ts_tablet_manager.cc:1428] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:38.800235 5783 heartbeater.cc:503] Master 127.20.115.254:36535 was elected leader, sending a full tablet report...
I20260924 21:20:38.800261 5932 raft_consensus.cc:348] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } }
I20260924 21:20:38.800431 5932 raft_consensus.cc:374] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:38.800465 5932 raft_consensus.cc:729] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04de93c244694b6d853d3554d552b893, State: Initialized, Role: FOLLOWER
I20260924 21:20:38.800401 5933 raft_consensus.cc:348] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } }
I20260924 21:20:38.800552 5933 raft_consensus.cc:374] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:38.800585 5933 raft_consensus.cc:729] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 576936364b98444e9b13805068b7f8c7, State: Initialized, Role: FOLLOWER
I20260924 21:20:38.800592 5932 consensus_queue.cc:260] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } }
I20260924 21:20:38.800688 5933 consensus_queue.cc:260] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } }
I20260924 21:20:38.800957 5933 ts_tablet_manager.cc:1428] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260924 21:20:38.801069 5649 heartbeater.cc:503] Master 127.20.115.254:36535 was elected leader, sending a full tablet report...
I20260924 21:20:38.800902 5932 ts_tablet_manager.cc:1428] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:20:38.800971 5917 heartbeater.cc:503] Master 127.20.115.254:36535 was elected leader, sending a full tablet report...
I20260924 21:20:38.838272 5939 raft_consensus.cc:482] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:38.838442 5939 raft_consensus.cc:504] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } }
I20260924 21:20:38.838829 5939 leader_election.cc:288] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 051824f623a44b27832e5bab0f363175 (127.20.115.194:41775), 04de93c244694b6d853d3554d552b893 (127.20.115.195:33105)
I20260924 21:20:38.842872 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.839133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:35545 (local address 127.20.115.194:41775)
0924 21:20:38.839201 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.839203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.839314 (+ 111us) server_negotiation.cc:407] Connection header received
0924 21:20:38.839362 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.839363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.839374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.839425 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:38.839693 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.840469 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.841927 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.842514 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.842593 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.842601 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.842769 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.842806 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:20:38.842871 5927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.839217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:41655 (local address 127.20.115.195:33105)
0924 21:20:38.839255 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.839257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.839264 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:20:38.839313 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.839314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.839325 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.839385 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:38.839693 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.840428 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.841927 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.842514 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.842592 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.842601 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.842768 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.842806 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:20:38.842912 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.839014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:33105 (local address 127.20.115.193:41655)
0924 21:20:38.839119 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.839132 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:38.839178 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:38.839529 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:38.839531 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:38.839532 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:38.839615 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.839617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.840507 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.840509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.841837 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.841840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.842605 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.842605 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.842685 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.842691 (+ 6us) client_negotiation.cc:769] Sending connection context
0924 21:20:38.842749 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 21:20:38.842803 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":58}
I20260924 21:20:38.842923 5940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.839016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:41775 (local address 127.20.115.193:35545)
0924 21:20:38.839241 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.839250 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:38.839302 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:38.839535 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:38.839536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:38.839537 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:38.839615 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.839617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.840525 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.840526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.841837 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:38.841840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.842605 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.842606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:38.842685 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.842692 (+ 7us) client_negotiation.cc:769] Sending connection context
0924 21:20:38.842749 (+ 57us) client_negotiation.cc:240] Negotiation successful
0924 21:20:38.842803 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":68,"thread_start_us":117,"threads_started":1}
I20260924 21:20:38.843297 5871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3a32281dc045ba8f8624f4b19990ae" candidate_uuid: "576936364b98444e9b13805068b7f8c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04de93c244694b6d853d3554d552b893" is_pre_election: true
I20260924 21:20:38.843324 5737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3a32281dc045ba8f8624f4b19990ae" candidate_uuid: "576936364b98444e9b13805068b7f8c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051824f623a44b27832e5bab0f363175" is_pre_election: true
I20260924 21:20:38.843456 5871 raft_consensus.cc:2454] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 576936364b98444e9b13805068b7f8c7 in term 0.
I20260924 21:20:38.843470 5737 raft_consensus.cc:2454] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 576936364b98444e9b13805068b7f8c7 in term 0.
I20260924 21:20:38.843686 5536 leader_election.cc:302] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051824f623a44b27832e5bab0f363175, 576936364b98444e9b13805068b7f8c7; no voters:
I20260924 21:20:38.843799 5939 raft_consensus.cc:2781] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:20:38.843852 5939 raft_consensus.cc:482] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:20:38.843874 5939 raft_consensus.cc:3037] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:38.844523 5939 raft_consensus.cc:504] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } }
I20260924 21:20:38.844688 5939 leader_election.cc:288] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [CANDIDATE]: Term 1 election: Requested vote from peers 051824f623a44b27832e5bab0f363175 (127.20.115.194:41775), 04de93c244694b6d853d3554d552b893 (127.20.115.195:33105)
W20260924 21:20:38.844728 5784 tablet.cc:2369] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:20:38.844842 5737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3a32281dc045ba8f8624f4b19990ae" candidate_uuid: "576936364b98444e9b13805068b7f8c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "051824f623a44b27832e5bab0f363175"
I20260924 21:20:38.844862 5871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f3a32281dc045ba8f8624f4b19990ae" candidate_uuid: "576936364b98444e9b13805068b7f8c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04de93c244694b6d853d3554d552b893"
I20260924 21:20:38.844924 5871 raft_consensus.cc:3037] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:38.844916 5737 raft_consensus.cc:3037] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:38.845614 5737 raft_consensus.cc:2454] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 576936364b98444e9b13805068b7f8c7 in term 1.
I20260924 21:20:38.845639 5871 raft_consensus.cc:2454] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 576936364b98444e9b13805068b7f8c7 in term 1.
I20260924 21:20:38.845782 5536 leader_election.cc:302] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 051824f623a44b27832e5bab0f363175, 576936364b98444e9b13805068b7f8c7; no voters:
I20260924 21:20:38.845884 5939 raft_consensus.cc:2781] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:38.846045 5939 raft_consensus.cc:686] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 1 LEADER]: Becoming Leader. State: Replica: 576936364b98444e9b13805068b7f8c7, State: Running, Role: LEADER
I20260924 21:20:38.846159 5939 consensus_queue.cc:237] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } }
I20260924 21:20:38.847054 5458 catalog_manager.cc:5495] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 576936364b98444e9b13805068b7f8c7 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "576936364b98444e9b13805068b7f8c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "576936364b98444e9b13805068b7f8c7" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 36945 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 } health_report { overall_health: UNKNOWN } } }
I20260924 21:20:38.867009 5928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.864659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55960 (local address 127.20.115.193:36945)
0924 21:20:38.864715 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:38.864716 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:38.864723 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:20:38.864752 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:38.864752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:38.864768 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:38.864903 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:20:38.865044 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.865503 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.866246 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:38.866440 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:38.866514 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:38.866671 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:20:38.866768 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:20:38.866859 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:38.866879 (+ 20us) server_negotiation.cc:299] Negotiation successful
0924 21:20:38.866898 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260924 21:20:38.868736 5871 raft_consensus.cc:1261] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [term 1 FOLLOWER]: Refusing update from remote peer 576936364b98444e9b13805068b7f8c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:38.868736 5737 raft_consensus.cc:1261] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [term 1 FOLLOWER]: Refusing update from remote peer 576936364b98444e9b13805068b7f8c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:38.869138 5939 consensus_queue.cc:1035] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04de93c244694b6d853d3554d552b893" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:38.869292 5941 consensus_queue.cc:1035] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "051824f623a44b27832e5bab0f363175" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:38.872236 5946 mvcc.cc:205] Tried to move back new op lower bound from 7333006700003811328 to 7333006699914514432. Current Snapshot: MvccSnapshot[applied={T|T < 7333006700003811328}]
I20260924 21:20:38.872241 5945 mvcc.cc:205] Tried to move back new op lower bound from 7333006700003811328 to 7333006699914514432. Current Snapshot: MvccSnapshot[applied={T|T < 7333006700003811328}]
I20260924 21:20:38.873737 5458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:38.873831 5458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32934:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:38.874580 5458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4759612137fd4281a64f7f3b84eb7122: Sending DeleteTablet for 3 replicas of tablet 2f3a32281dc045ba8f8624f4b19990ae
I20260924 21:20:38.874873 5583 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f3a32281dc045ba8f8624f4b19990ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:38 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:55948
I20260924 21:20:38.874902 5851 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f3a32281dc045ba8f8624f4b19990ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:38 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:36764
I20260924 21:20:38.874933 5717 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f3a32281dc045ba8f8624f4b19990ae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:38 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:41580
I20260924 21:20:38.875149 5953 tablet_replica.cc:330] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7: stopping tablet replica
I20260924 21:20:38.875228 5955 tablet_replica.cc:330] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175: stopping tablet replica
I20260924 21:20:38.875272 5953 raft_consensus.cc:2229] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:20:38.875294 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 5520
I20260924 21:20:38.875382 5953 raft_consensus.cc:2258] T 2f3a32281dc045ba8f8624f4b19990ae P 576936364b98444e9b13805068b7f8c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:38.875519 5954 tablet_replica.cc:330] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: stopping tablet replica
I20260924 21:20:38.875600 5954 raft_consensus.cc:2229] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:38.875679 5954 raft_consensus.cc:2258] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:38.876083 5954 ts_tablet_manager.cc:1905] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:38.876567 5955 raft_consensus.cc:2229] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:38.876680 5955 raft_consensus.cc:2258] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:38.877820 5954 ts_tablet_manager.cc:1918] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:38.877931 5954 log.cc:1196] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/2f3a32281dc045ba8f8624f4b19990ae
I20260924 21:20:38.878193 5954 ts_tablet_manager.cc:1939] T 2f3a32281dc045ba8f8624f4b19990ae P 04de93c244694b6d853d3554d552b893: Deleting consensus metadata
I20260924 21:20:38.878597 5445 catalog_manager.cc:4841] TS 04de93c244694b6d853d3554d552b893 (127.20.115.195:33105): tablet 2f3a32281dc045ba8f8624f4b19990ae (table test-table [id=3942270b6f1d4db88a72b7824b07a208]) successfully deleted
I20260924 21:20:38.881135 5955 ts_tablet_manager.cc:1905] T 2f3a32281dc045ba8f8624f4b19990ae P 051824f623a44b27832e5bab0f363175: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260924 21:20:38.881260 5442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:36945 (error 108)
I20260924 21:20:38.881505 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 5654
I20260924 21:20:38.882095 5926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.881645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:36945 (local address 127.0.0.1:55974)
0924 21:20:38.882020 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.882062 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:36945: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353}
W20260924 21:20:38.882216 5442 catalog_manager.cc:4584] TS 576936364b98444e9b13805068b7f8c7 (127.20.115.193:36945): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2f3a32281dc045ba8f8624f4b19990ae: Network error: Client connection negotiation failed: client connection to 127.20.115.193:36945: connect: Connection refused (error 111)
I20260924 21:20:38.886710 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 5788
I20260924 21:20:38.886740 5926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:38.886581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:41775 (local address 127.0.0.1:41582)
0924 21:20:38.886664 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:38.886702 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.194:41775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260924 21:20:38.891232 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 5426
2026-09-24T21:20:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (1892 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-24T21:20:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:38Z Disabled control of system clock
I20260924 21:20:38.918159 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:40001
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:40001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:20:38.995919 5960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:38.996114 5960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:38.996133 5960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:38.997293 5960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:38.997331 5960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:38.997344 5960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:38.997357 5960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:38.997370 5960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:38.998675 5960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:40001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:40001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:38.998914 5960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:38.999194 5960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:39.003850 5965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:39.003862 5968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:39.003862 5966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:39.003880 5960 server_base.cc:1034] running on GCE node
I20260924 21:20:39.004324 5960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:39.004606 5960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:39.005748 5960 hybrid_clock.cc:648] HybridClock initialized: now 1790284839005726 us; error 38 us; skew 500 ppm
I20260924 21:20:39.007010 5960 webserver.cc:460] Webserver started at http://127.20.115.254:41893/ using document root <none> and password file <none>
I20260924 21:20:39.007297 5960 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:39.007345 5960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:39.007498 5960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:39.008558 5960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "91fedc11ff7a4c6ba2551c9116cf1407"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "892ea1cc9950c8d91b1e3d9ab36bf3c3"
server_key_iv: "f81a8cb3f7d8b520cbbe36ea4c76e6e5"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.008932 5960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "91fedc11ff7a4c6ba2551c9116cf1407"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "892ea1cc9950c8d91b1e3d9ab36bf3c3"
server_key_iv: "f81a8cb3f7d8b520cbbe36ea4c76e6e5"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.010171 5960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:20:39.010835 5974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.010991 5960 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:20:39.011099 5960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "91fedc11ff7a4c6ba2551c9116cf1407"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "892ea1cc9950c8d91b1e3d9ab36bf3c3"
server_key_iv: "f81a8cb3f7d8b520cbbe36ea4c76e6e5"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.011171 5960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:39.045115 5960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:39.045449 5960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:39.045610 5960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:39.049582 5960 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:40001
I20260924 21:20:39.049613 6026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:40001 every 8 connection(s)
I20260924 21:20:39.049966 5960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:39.050887 6027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:39.052250 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 5960
I20260924 21:20:39.052349 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:39.052531 20943 external_mini_cluster.cc:1428] Setting key a3048be6b37ae2f3313417b09941d9e9
I20260924 21:20:39.053546 6027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407: Bootstrap starting.
I20260924 21:20:39.054091 6027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:39.054400 6027 log.cc:824] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:39.055053 6027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407: No bootstrap required, opened a new log
I20260924 21:20:39.057389 6027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fedc11ff7a4c6ba2551c9116cf1407" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40001 } }
I20260924 21:20:39.057565 6027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:39.057611 6027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91fedc11ff7a4c6ba2551c9116cf1407, State: Initialized, Role: FOLLOWER
I20260924 21:20:39.057732 6027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fedc11ff7a4c6ba2551c9116cf1407" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40001 } }
I20260924 21:20:39.057823 6027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:39.057866 6027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:39.057911 6027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:39.058177 6030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.053058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42182 (local address 127.20.115.254:40001)
0924 21:20:39.053275 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.053284 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.053316 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:20:39.053786 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.053793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.053859 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.054008 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:39.054227 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.055158 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.056255 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.056644 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.056733 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.057436 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:39.057439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:39.057440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:39.057628 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:39.057676 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:39.057690 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:39.057777 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.057904 (+ 127us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.057994 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260924 21:20:39.058534 6027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fedc11ff7a4c6ba2551c9116cf1407" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40001 } }
I20260924 21:20:39.058637 6027 leader_election.cc:302] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91fedc11ff7a4c6ba2551c9116cf1407; no voters:
I20260924 21:20:39.058784 6027 leader_election.cc:288] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:39.058818 6032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:39.058983 6032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [term 1 LEADER]: Becoming Leader. State: Replica: 91fedc11ff7a4c6ba2551c9116cf1407, State: Running, Role: LEADER
I20260924 21:20:39.059006 6027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:39.059077 6032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fedc11ff7a4c6ba2551c9116cf1407" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40001 } }
I20260924 21:20:39.059448 6033 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91fedc11ff7a4c6ba2551c9116cf1407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fedc11ff7a4c6ba2551c9116cf1407" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40001 } } }
I20260924 21:20:39.059589 6033 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:39.059449 6034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91fedc11ff7a4c6ba2551c9116cf1407. Latest consensus state: current_term: 1 leader_uuid: "91fedc11ff7a4c6ba2551c9116cf1407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91fedc11ff7a4c6ba2551c9116cf1407" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 40001 } } }
I20260924 21:20:39.059758 6034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:39.059994 6041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:39.060439 6041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:39.061923 6041 catalog_manager.cc:1347] Generated new cluster ID: 1828241cc52047228e144be1af805328
I20260924 21:20:39.061991 6041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:39.066471 6041 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:39.067143 6041 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:39.089865 6041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407: Generated new TSK 0
I20260924 21:20:39.090101 6041 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:39.093032 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40001
--builtin_ntp_servers=127.20.115.212:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:20:39.174752 6051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:39.174960 6051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:39.174983 6051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:39.176179 6051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:39.176221 6051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:39.176246 6051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:20:39.177832 6051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:39.178079 6051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:39.178388 6051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:39.183398 6056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:39.183455 6059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:39.183473 6057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:39.183699 6051 server_base.cc:1034] running on GCE node
I20260924 21:20:39.183887 6051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:39.184160 6051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:39.185322 6051 hybrid_clock.cc:648] HybridClock initialized: now 1790284839185316 us; error 38 us; skew 500 ppm
I20260924 21:20:39.186625 6051 webserver.cc:460] Webserver started at http://127.20.115.193:42855/ using document root <none> and password file <none>
I20260924 21:20:39.186879 6051 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:39.186947 6051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:39.187100 6051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:39.188230 6051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "2e6e705749414be99f4297e8549f5396"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "12ae8d269b4ecd721e3e7872b4c98b5a"
server_key_iv: "04b2df7c98652282df1587c436875be5"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.188581 6051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "2e6e705749414be99f4297e8549f5396"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "12ae8d269b4ecd721e3e7872b4c98b5a"
server_key_iv: "04b2df7c98652282df1587c436875be5"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.190034 6051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:20:39.190860 6065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.191058 6051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:39.191154 6051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "2e6e705749414be99f4297e8549f5396"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "12ae8d269b4ecd721e3e7872b4c98b5a"
server_key_iv: "04b2df7c98652282df1587c436875be5"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.191252 6051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:39.199934 6051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:39.200225 6051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:39.200392 6051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:39.200595 6051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:39.200904 6051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:39.200938 6051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.200959 6051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:39.200971 6051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.206609 6051 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:44255
I20260924 21:20:39.206653 6178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:44255 every 8 connection(s)
I20260924 21:20:39.206951 6051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:20:39.208034 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 6051
I20260924 21:20:39.208143 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:20:39.208343 20943 external_mini_cluster.cc:1428] Setting key 3884a70cb164e758341452589ee3a170
I20260924 21:20:39.209291 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40001
--builtin_ntp_servers=127.20.115.212:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 21:20:39.212560 6030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.207667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:54245 (local address 127.20.115.254:40001)
0924 21:20:39.207731 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.207733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.207957 (+ 224us) server_negotiation.cc:407] Connection header received
0924 21:20:39.208662 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.208663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.208678 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.208745 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:39.209605 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.210100 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.211080 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.211378 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.211465 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.212282 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:39.212284 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:39.212284 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:39.212288 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:39.212303 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:39.212306 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:39.212355 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.212460 (+ 105us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.212486 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:20:39.212766 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.207642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40001 (local address 127.20.115.193:54245)
0924 21:20:39.207845 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:39.207876 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:39.208560 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:39.208828 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:39.208832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:39.208842 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:39.209496 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.209502 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.210201 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.210203 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.210974 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.210975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.211475 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.211476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.211614 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.212135 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:20:39.212135 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:39.212166 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:20:39.212185 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:39.212369 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:39.212373 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:39.212376 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:39.212462 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:20:39.212558 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
I20260924 21:20:39.213264 6179 heartbeater.cc:347] Connected to a master server at 127.20.115.254:40001
I20260924 21:20:39.213351 6179 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:39.213539 6179 heartbeater.cc:511] Master 127.20.115.254:40001 requested a full tablet report, sending...
I20260924 21:20:39.214040 5991 ts_manager.cc:194] Registered new tserver with Master: 2e6e705749414be99f4297e8549f5396 (127.20.115.193:44255)
I20260924 21:20:39.214597 5991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.193:54245
W20260924 21:20:39.289023 6182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:39.289234 6182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:39.289269 6182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:39.290357 6182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:39.290416 6182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:39.290457 6182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:20:39.291779 6182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:39.292032 6182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:39.292307 6182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:39.297261 6190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:39.297281 6188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:39.297262 6187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:39.297456 6182 server_base.cc:1034] running on GCE node
I20260924 21:20:39.297613 6182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:39.297797 6182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:39.298955 6182 hybrid_clock.cc:648] HybridClock initialized: now 1790284839298936 us; error 35 us; skew 500 ppm
I20260924 21:20:39.300239 6182 webserver.cc:460] Webserver started at http://127.20.115.194:33691/ using document root <none> and password file <none>
I20260924 21:20:39.300465 6182 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:39.300544 6182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:39.300684 6182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:39.301755 6182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "529b2945dec24fd1a58bb8ea5f8891c4"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "25c47c0e99e6946d27ae25ff6d979d78"
server_key_iv: "a94c738e9f0a4fb61ac0691a991f2fec"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.302094 6182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "529b2945dec24fd1a58bb8ea5f8891c4"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "25c47c0e99e6946d27ae25ff6d979d78"
server_key_iv: "a94c738e9f0a4fb61ac0691a991f2fec"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.303377 6182 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:20:39.304153 6196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.304322 6182 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:39.304415 6182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "529b2945dec24fd1a58bb8ea5f8891c4"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "25c47c0e99e6946d27ae25ff6d979d78"
server_key_iv: "a94c738e9f0a4fb61ac0691a991f2fec"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.304502 6182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:39.322614 6182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:39.322882 6182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:39.323028 6182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:39.323252 6182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:39.323561 6182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:39.323617 6182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.323657 6182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:39.323683 6182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.329350 6182 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:44537
I20260924 21:20:39.329418 6309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:44537 every 8 connection(s)
I20260924 21:20:39.329748 6182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:20:39.333809 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 6182
I20260924 21:20:39.333945 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:20:39.334162 20943 external_mini_cluster.cc:1428] Setting key 0fee5624b3ccbe470d840fd547bdb752
I20260924 21:20:39.334829 6312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.330443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40001 (local address 127.20.115.194:48461)
0924 21:20:39.330615 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:39.330654 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:39.331096 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:39.331295 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:39.331298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:39.331306 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:39.331765 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.331768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.332428 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.332429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.333207 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.333209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.333549 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.333550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.333648 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.334204 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:20:39.334204 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:39.334260 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:20:39.334287 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:39.334449 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:39.334453 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:39.334456 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:39.334528 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:20:39.334622 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260924 21:20:39.334820 6030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.330492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:48461 (local address 127.20.115.254:40001)
0924 21:20:39.330541 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.330543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.330745 (+ 202us) server_negotiation.cc:407] Connection header received
0924 21:20:39.331181 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.331182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.331192 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.331250 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:39.331867 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.332349 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.333287 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.333468 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.333524 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.334366 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:39.334368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:39.334368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:39.334373 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:39.334388 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:39.334392 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:39.334446 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.334729 (+ 283us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.334752 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:20:39.335243 6310 heartbeater.cc:347] Connected to a master server at 127.20.115.254:40001
I20260924 21:20:39.335340 6310 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:39.335382 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40001
--builtin_ntp_servers=127.20.115.212:38357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 21:20:39.335537 6310 heartbeater.cc:511] Master 127.20.115.254:40001 requested a full tablet report, sending...
I20260924 21:20:39.335868 5991 ts_manager.cc:194] Registered new tserver with Master: 529b2945dec24fd1a58bb8ea5f8891c4 (127.20.115.194:44537)
I20260924 21:20:39.336279 5991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.194:48461
W20260924 21:20:39.416468 6313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:39.416668 6313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:39.416690 6313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:39.417816 6313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:39.417855 6313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:39.417879 6313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:20:39.419131 6313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:38357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:40001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:39.419314 6313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:39.419602 6313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:39.424496 6319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:39.424474 6318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:39.424499 6321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:39.424757 6313 server_base.cc:1034] running on GCE node
I20260924 21:20:39.424978 6313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:39.425205 6313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:39.426362 6313 hybrid_clock.cc:648] HybridClock initialized: now 1790284839426354 us; error 37 us; skew 500 ppm
I20260924 21:20:39.427536 6313 webserver.cc:460] Webserver started at http://127.20.115.195:45523/ using document root <none> and password file <none>
I20260924 21:20:39.427788 6313 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:39.427856 6313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:39.428004 6313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:39.429077 6313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "90466bcec88b42439767a410a7b8a838"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "09e39a58fca9aaa841be5a0e7bd0b64d"
server_key_iv: "dcb584cabe70050d8da2f50bae1f4fef"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.429414 6313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "90466bcec88b42439767a410a7b8a838"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "09e39a58fca9aaa841be5a0e7bd0b64d"
server_key_iv: "dcb584cabe70050d8da2f50bae1f4fef"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.430691 6313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:39.431396 6327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.431612 6313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:39.431717 6313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "90466bcec88b42439767a410a7b8a838"
format_stamp: "Formatted at 2026-09-24 21:20:39 on dist-test-slave-cxpd"
server_key: "09e39a58fca9aaa841be5a0e7bd0b64d"
server_key_iv: "dcb584cabe70050d8da2f50bae1f4fef"
server_key_version: "encryptionkey@0"
I20260924 21:20:39.431797 6313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:39.453696 6313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:39.453982 6313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:39.454130 6313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:39.454349 6313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:39.454704 6313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:39.454763 6313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.454797 6313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:39.454824 6313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:39.460810 6313 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:34239
I20260924 21:20:39.460846 6440 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:34239 every 8 connection(s)
I20260924 21:20:39.461153 6313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:20:39.466189 6030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.461819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:32963 (local address 127.20.115.254:40001)
0924 21:20:39.461861 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.461863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.462064 (+ 201us) server_negotiation.cc:407] Connection header received
0924 21:20:39.462573 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.462574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.462586 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.462649 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:39.463299 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.463794 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.464769 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.465001 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.465085 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.465891 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:39.465893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:39.465893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:39.465897 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:39.465912 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:39.465916 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:39.465983 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.466079 (+ 96us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.466105 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:20:39.466382 6443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.461787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:40001 (local address 127.20.115.195:32963)
0924 21:20:39.461950 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:39.461981 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:39.462469 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:39.462722 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:39.462726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:39.462735 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:39.463206 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.463210 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.463881 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.463882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.464672 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.464674 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.465095 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.465096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.465232 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.465753 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:20:39.465753 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:39.465779 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:20:39.465800 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:39.465983 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:39.465989 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:39.465992 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:39.466082 (+ 90us) client_negotiation.cc:240] Negotiation successful
0924 21:20:39.466177 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
I20260924 21:20:39.466868 6441 heartbeater.cc:347] Connected to a master server at 127.20.115.254:40001
I20260924 21:20:39.466949 6441 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:39.467087 6441 heartbeater.cc:511] Master 127.20.115.254:40001 requested a full tablet report, sending...
I20260924 21:20:39.467428 5991 ts_manager.cc:194] Registered new tserver with Master: 90466bcec88b42439767a410a7b8a838 (127.20.115.195:34239)
I20260924 21:20:39.467770 5991 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.195:32963
I20260924 21:20:39.470335 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 6313
I20260924 21:20:39.470422 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:20:39.470575 20943 external_mini_cluster.cc:1428] Setting key 23c9b072d68380826b94702451fa9c67
I20260924 21:20:39.471406 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:20:39.475383 6030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.472764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42184 (local address 127.20.115.254:40001)
0924 21:20:39.472831 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.472833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.472919 (+ 86us) server_negotiation.cc:407] Connection header received
0924 21:20:39.472998 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.472999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.473007 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.473047 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:39.473158 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.473602 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.474329 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.474496 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.474544 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.475214 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:39.475215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:39.475215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:39.475218 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:39.475227 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:39.475230 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:39.475266 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.475318 (+ 52us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.475333 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:20:39.476961 5991 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:20:39.478533 5991 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:20:39.484727 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.480701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59702 (local address 127.20.115.195:34239)
0924 21:20:39.481131 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.481135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.481142 (+ 7us) server_negotiation.cc:407] Connection header received
0924 21:20:39.481190 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.481193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.481207 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.481289 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:39.481438 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.482226 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.483841 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.484304 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.484389 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.484470 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.484599 (+ 129us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.484634 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":99,"threads_started":1}
I20260924 21:20:39.484737 6451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.480544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:34239 (local address 127.0.0.1:59702)
0924 21:20:39.480936 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:39.480945 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:39.480982 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:39.481303 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:39.481304 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:39.481307 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:39.481354 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.481356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.482312 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.482313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.483769 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.483770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.484401 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.484402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.484478 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.484491 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:20:39.484584 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 21:20:39.484636 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":32,"thread_start_us":73,"threads_started":1}
I20260924 21:20:39.484817 6449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.480539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:44255 (local address 127.0.0.1:48904)
0924 21:20:39.480697 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:39.480734 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:39.480830 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:39.481216 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:39.481219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:39.481221 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:39.481285 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.481287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.482562 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.482563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.483654 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.483656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.484470 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.484471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.484547 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.484551 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:39.484612 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:20:39.484659 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":60,"threads_started":1}
I20260924 21:20:39.485044 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.480821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57246 (local address 127.20.115.194:44537)
0924 21:20:39.480979 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.480983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.481492 (+ 509us) server_negotiation.cc:407] Connection header received
0924 21:20:39.481535 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.481538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.481553 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.481626 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:39.481721 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.482715 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.483635 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.484369 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.484443 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.484524 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.484908 (+ 384us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.484952 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":96,"threads_started":1}
I20260924 21:20:39.485097 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.480887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48904 (local address 127.20.115.193:44255)
0924 21:20:39.481059 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.481062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.481070 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:20:39.481103 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.481105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.481116 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.481189 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:39.481373 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.482479 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.483725 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.484374 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.484738 (+ 364us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.484954 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.484977 (+ 23us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.485012 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":98,"threads_started":1}
I20260924 21:20:39.485204 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.480561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:44537 (local address 127.0.0.1:57246)
0924 21:20:39.481376 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:39.481386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:39.481433 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:39.481624 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:39.481624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:39.481627 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:39.481659 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.481661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.482800 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.482801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.483539 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.483541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.484557 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.484558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.484615 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.484619 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:39.484663 (+ 44us) client_negotiation.cc:240] Negotiation successful
0924 21:20:39.484695 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"thread_start_us":117,"threads_started":1}
I20260924 21:20:39.485703 6113 tablet_service.cc:1467] Processing CreateTablet for tablet c6750fd304fc4dba9e36aad3783664c4 (DEFAULT_TABLE table=test-table [id=fb2d7cde30ed40219142fd9e55bfba34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:39.485809 6375 tablet_service.cc:1467] Processing CreateTablet for tablet c6750fd304fc4dba9e36aad3783664c4 (DEFAULT_TABLE table=test-table [id=fb2d7cde30ed40219142fd9e55bfba34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:39.485849 6244 tablet_service.cc:1467] Processing CreateTablet for tablet c6750fd304fc4dba9e36aad3783664c4 (DEFAULT_TABLE table=test-table [id=fb2d7cde30ed40219142fd9e55bfba34]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:39.486063 6113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6750fd304fc4dba9e36aad3783664c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:39.486135 6244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6750fd304fc4dba9e36aad3783664c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:39.486155 6375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6750fd304fc4dba9e36aad3783664c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:39.488672 6455 tablet_bootstrap.cc:492] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: Bootstrap starting.
I20260924 21:20:39.489274 6456 tablet_bootstrap.cc:492] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: Bootstrap starting.
I20260924 21:20:39.489305 6455 tablet_bootstrap.cc:654] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:39.489621 6455 log.cc:824] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:39.489892 6456 tablet_bootstrap.cc:654] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:39.490175 6456 log.cc:824] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:39.490406 6455 tablet_bootstrap.cc:492] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: No bootstrap required, opened a new log
I20260924 21:20:39.490475 6455 ts_tablet_manager.cc:1397] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:20:39.490483 6457 tablet_bootstrap.cc:492] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396: Bootstrap starting.
I20260924 21:20:39.490823 6456 tablet_bootstrap.cc:492] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: No bootstrap required, opened a new log
I20260924 21:20:39.490906 6456 ts_tablet_manager.cc:1397] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:20:39.491088 6457 tablet_bootstrap.cc:654] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:39.491452 6457 log.cc:824] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:39.492363 6457 tablet_bootstrap.cc:492] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396: No bootstrap required, opened a new log
I20260924 21:20:39.492437 6457 ts_tablet_manager.cc:1397] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:20:39.492941 6455 raft_consensus.cc:348] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } }
I20260924 21:20:39.493089 6455 raft_consensus.cc:374] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:39.493130 6455 raft_consensus.cc:729] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90466bcec88b42439767a410a7b8a838, State: Initialized, Role: FOLLOWER
I20260924 21:20:39.493258 6455 consensus_queue.cc:260] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } }
I20260924 21:20:39.493533 6441 heartbeater.cc:503] Master 127.20.115.254:40001 was elected leader, sending a full tablet report...
I20260924 21:20:39.493546 6455 ts_tablet_manager.cc:1428] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:20:39.493713 6456 raft_consensus.cc:348] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } }
I20260924 21:20:39.493886 6456 raft_consensus.cc:374] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:39.493928 6456 raft_consensus.cc:729] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529b2945dec24fd1a58bb8ea5f8891c4, State: Initialized, Role: FOLLOWER
I20260924 21:20:39.494059 6456 consensus_queue.cc:260] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } }
I20260924 21:20:39.494347 6456 ts_tablet_manager.cc:1428] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260924 21:20:39.494421 6310 heartbeater.cc:503] Master 127.20.115.254:40001 was elected leader, sending a full tablet report...
I20260924 21:20:39.495139 6457 raft_consensus.cc:348] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } }
I20260924 21:20:39.495273 6457 raft_consensus.cc:374] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:39.495326 6457 raft_consensus.cc:729] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e6e705749414be99f4297e8549f5396, State: Initialized, Role: FOLLOWER
I20260924 21:20:39.495436 6457 consensus_queue.cc:260] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } }
I20260924 21:20:39.495702 6457 ts_tablet_manager.cc:1428] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260924 21:20:39.495735 6179 heartbeater.cc:503] Master 127.20.115.254:40001 was elected leader, sending a full tablet report...
I20260924 21:20:39.520767 6463 raft_consensus.cc:482] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:39.520953 6463 raft_consensus.cc:504] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } }
I20260924 21:20:39.521376 6463 leader_election.cc:288] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90466bcec88b42439767a410a7b8a838 (127.20.115.195:34239), 529b2945dec24fd1a58bb8ea5f8891c4 (127.20.115.194:44537)
I20260924 21:20:39.524091 6453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.521624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:55631 (local address 127.20.115.194:44537)
0924 21:20:39.521671 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.521674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.521684 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:20:39.521713 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.521715 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.521724 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.521811 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:39.521975 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.522460 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.523390 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.523751 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.523838 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.523846 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.523990 (+ 144us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.524024 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:20:39.524144 6452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.521657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:46059 (local address 127.20.115.195:34239)
0924 21:20:39.521700 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.521702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.521761 (+ 59us) server_negotiation.cc:407] Connection header received
0924 21:20:39.521824 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.521825 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.521834 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.521883 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:39.522042 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.522496 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.523420 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.523784 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.523869 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.523876 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.524041 (+ 165us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.524076 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:20:39.524183 6464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.521553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:34239 (local address 127.20.115.193:46059)
0924 21:20:39.521718 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:39.521726 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:39.521771 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:39.521885 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:39.521886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:39.521887 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:39.521958 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.521959 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.522577 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.522578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.523327 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.523328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.523869 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.523870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.523949 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.523959 (+ 10us) client_negotiation.cc:769] Sending connection context
0924 21:20:39.524033 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 21:20:39.524069 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":8,"thread_start_us":116,"threads_started":1}
I20260924 21:20:39.524470 6181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.521523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:44537 (local address 127.20.115.193:55631)
0924 21:20:39.521566 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:39.521579 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:39.521624 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:39.521816 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:39.521818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:39.521819 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:39.521896 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.521898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.522547 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.522548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.523295 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:39.523297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.523838 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.523838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:39.523911 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.523914 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:39.523983 (+ 69us) client_negotiation.cc:240] Negotiation successful
0924 21:20:39.524024 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260924 21:20:39.524442 6395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6750fd304fc4dba9e36aad3783664c4" candidate_uuid: "2e6e705749414be99f4297e8549f5396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90466bcec88b42439767a410a7b8a838" is_pre_election: true
I20260924 21:20:39.524572 6395 raft_consensus.cc:2454] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e6e705749414be99f4297e8549f5396 in term 0.
I20260924 21:20:39.524763 6068 leader_election.cc:302] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e6e705749414be99f4297e8549f5396, 90466bcec88b42439767a410a7b8a838; no voters:
I20260924 21:20:39.524858 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6750fd304fc4dba9e36aad3783664c4" candidate_uuid: "2e6e705749414be99f4297e8549f5396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" is_pre_election: true
I20260924 21:20:39.524933 6463 raft_consensus.cc:2781] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:20:39.524987 6463 raft_consensus.cc:482] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:20:39.525015 6463 raft_consensus.cc:3037] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:39.525002 6264 raft_consensus.cc:2454] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e6e705749414be99f4297e8549f5396 in term 0.
I20260924 21:20:39.525866 6463 raft_consensus.cc:504] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } }
I20260924 21:20:39.525998 6463 leader_election.cc:288] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [CANDIDATE]: Term 1 election: Requested vote from peers 90466bcec88b42439767a410a7b8a838 (127.20.115.195:34239), 529b2945dec24fd1a58bb8ea5f8891c4 (127.20.115.194:44537)
I20260924 21:20:39.526142 6395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6750fd304fc4dba9e36aad3783664c4" candidate_uuid: "2e6e705749414be99f4297e8549f5396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90466bcec88b42439767a410a7b8a838"
I20260924 21:20:39.526165 6264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6750fd304fc4dba9e36aad3783664c4" candidate_uuid: "2e6e705749414be99f4297e8549f5396" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "529b2945dec24fd1a58bb8ea5f8891c4"
I20260924 21:20:39.526221 6395 raft_consensus.cc:3037] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:39.526261 6264 raft_consensus.cc:3037] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:39.526897 6395 raft_consensus.cc:2454] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e6e705749414be99f4297e8549f5396 in term 1.
I20260924 21:20:39.527050 6068 leader_election.cc:302] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e6e705749414be99f4297e8549f5396, 90466bcec88b42439767a410a7b8a838; no voters:
I20260924 21:20:39.527083 6264 raft_consensus.cc:2454] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e6e705749414be99f4297e8549f5396 in term 1.
I20260924 21:20:39.527160 6463 raft_consensus.cc:2781] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:39.527315 6463 raft_consensus.cc:686] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [term 1 LEADER]: Becoming Leader. State: Replica: 2e6e705749414be99f4297e8549f5396, State: Running, Role: LEADER
I20260924 21:20:39.527416 6463 consensus_queue.cc:237] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } }
I20260924 21:20:39.528302 5990 catalog_manager.cc:5495] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e6e705749414be99f4297e8549f5396 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "2e6e705749414be99f4297e8549f5396" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e6e705749414be99f4297e8549f5396" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 44255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 } health_report { overall_health: UNKNOWN } } }
I20260924 21:20:39.541891 6454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.539462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48920 (local address 127.20.115.193:44255)
0924 21:20:39.539500 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:39.539503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:39.539528 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:20:39.539562 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:39.539562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:39.539581 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:39.539670 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:20:39.539782 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.540258 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.541101 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:39.541313 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:39.541390 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:39.541550 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:20:39.541655 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:20:39.541750 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:39.541774 (+ 24us) server_negotiation.cc:299] Negotiation successful
0924 21:20:39.541795 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:20:39.543592 6395 raft_consensus.cc:1261] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [term 1 FOLLOWER]: Refusing update from remote peer 2e6e705749414be99f4297e8549f5396: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:39.543592 6264 raft_consensus.cc:1261] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [term 1 FOLLOWER]: Refusing update from remote peer 2e6e705749414be99f4297e8549f5396: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:39.543928 6463 consensus_queue.cc:1035] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "529b2945dec24fd1a58bb8ea5f8891c4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:39.544098 6465 consensus_queue.cc:1035] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90466bcec88b42439767a410a7b8a838" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 34239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:39.548640 5990 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:39.548734 5990 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:39.549537 5990 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 91fedc11ff7a4c6ba2551c9116cf1407: Sending DeleteTablet for 3 replicas of tablet c6750fd304fc4dba9e36aad3783664c4
I20260924 21:20:39.549878 6244 tablet_service.cc:1514] Processing DeleteTablet for tablet c6750fd304fc4dba9e36aad3783664c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:39 UTC) from {username='slave'} at 127.0.0.1:57246
I20260924 21:20:39.549906 6375 tablet_service.cc:1514] Processing DeleteTablet for tablet c6750fd304fc4dba9e36aad3783664c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:39 UTC) from {username='slave'} at 127.0.0.1:59702
I20260924 21:20:39.550165 6478 tablet_replica.cc:330] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: stopping tablet replica
I20260924 21:20:39.550150 6113 tablet_service.cc:1514] Processing DeleteTablet for tablet c6750fd304fc4dba9e36aad3783664c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:39 UTC) from {username='slave'} at 127.0.0.1:48904
I20260924 21:20:39.550257 6478 raft_consensus.cc:2229] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:39.550335 6480 tablet_replica.cc:330] T c6750fd304fc4dba9e36aad3783664c4 P 2e6e705749414be99f4297e8549f5396: stopping tablet replica
I20260924 21:20:39.550330 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 6051
I20260924 21:20:39.550347 6478 raft_consensus.cc:2258] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:39.550786 6478 ts_tablet_manager.cc:1905] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:39.551906 6479 tablet_replica.cc:330] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: stopping tablet replica
I20260924 21:20:39.551975 6479 raft_consensus.cc:2229] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:39.552048 6479 raft_consensus.cc:2258] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:39.552357 6478 ts_tablet_manager.cc:1918] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:39.552415 6478 log.cc:1196] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/c6750fd304fc4dba9e36aad3783664c4
I20260924 21:20:39.552413 6479 ts_tablet_manager.cc:1905] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:39.552687 6478 ts_tablet_manager.cc:1939] T c6750fd304fc4dba9e36aad3783664c4 P 529b2945dec24fd1a58bb8ea5f8891c4: Deleting consensus metadata
I20260924 21:20:39.553074 5976 catalog_manager.cc:4841] TS 529b2945dec24fd1a58bb8ea5f8891c4 (127.20.115.194:44537): tablet c6750fd304fc4dba9e36aad3783664c4 (table test-table [id=fb2d7cde30ed40219142fd9e55bfba34]) successfully deleted
I20260924 21:20:39.554127 6479 ts_tablet_manager.cc:1918] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:39.554183 6479 log.cc:1196] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/c6750fd304fc4dba9e36aad3783664c4
I20260924 21:20:39.554415 6479 ts_tablet_manager.cc:1939] T c6750fd304fc4dba9e36aad3783664c4 P 90466bcec88b42439767a410a7b8a838: Deleting consensus metadata
I20260924 21:20:39.554773 5977 catalog_manager.cc:4841] TS 90466bcec88b42439767a410a7b8a838 (127.20.115.195:34239): tablet c6750fd304fc4dba9e36aad3783664c4 (table test-table [id=fb2d7cde30ed40219142fd9e55bfba34]) successfully deleted
W20260924 21:20:39.556542 5975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:44255 (error 108)
I20260924 21:20:39.556747 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 6182
I20260924 21:20:39.557021 6450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:39.556913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:44255 (local address 127.0.0.1:48932)
0924 21:20:39.556958 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:39.556990 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:44255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260924 21:20:39.557121 5975 catalog_manager.cc:4584] TS 2e6e705749414be99f4297e8549f5396 (127.20.115.193:44255): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6750fd304fc4dba9e36aad3783664c4: Network error: Client connection negotiation failed: client connection to 127.20.115.193:44255: connect: Connection refused (error 111)
I20260924 21:20:39.561965 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 6313
I20260924 21:20:39.566845 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 5960
2026-09-24T21:20:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (674 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:39 dist-test-slave-cxpd krb5kdc[6484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:39 dist-test-slave-cxpd krb5kdc[6484](info): set up 2 sockets
Sep 24 21:20:39 dist-test-slave-cxpd krb5kdc[6484](info): commencing operation
krb5kdc: starting...
W20260924 21:20:40.616592 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:40 dist-test-slave-cxpd krb5kdc[6484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:20:40.736866 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43559
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:43559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:40.815624 6497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:40.815815 6497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:40.815836 6497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:40.816916 6497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:40.816954 6497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:40.816968 6497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:40.816980 6497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:40.816994 6497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:40.818382 6497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:43559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:40.818563 6497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:40.818732 6497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:40.823424 6503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:40.823424 6505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:40.823475 6502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:40.823612 6497 server_base.cc:1034] running on GCE node
I20260924 21:20:40.823988 6497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:40.824296 6497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:40.825448 6497 hybrid_clock.cc:648] HybridClock initialized: now 1790284840825435 us; error 29 us; skew 500 ppm
Sep 24 21:20:40 dist-test-slave-cxpd krb5kdc[6484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:40.827924 6497 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:20:40.828332 6497 webserver.cc:460] Webserver started at http://127.20.115.254:42405/ using document root <none> and password file <none>
I20260924 21:20:40.828567 6497 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:40.828635 6497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:40.828835 6497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:40.829859 6497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "a3625d8c60a340faabc944133b680bb9"
format_stamp: "Formatted at 2026-09-24 21:20:40 on dist-test-slave-cxpd"
server_key: "f103fd46a3d359b3b6e0ad57dd9081b0"
server_key_iv: "378bbb285de14c8bfd76f514767dfc41"
server_key_version: "encryptionkey@0"
I20260924 21:20:40.830215 6497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "a3625d8c60a340faabc944133b680bb9"
format_stamp: "Formatted at 2026-09-24 21:20:40 on dist-test-slave-cxpd"
server_key: "f103fd46a3d359b3b6e0ad57dd9081b0"
server_key_iv: "378bbb285de14c8bfd76f514767dfc41"
server_key_version: "encryptionkey@0"
I20260924 21:20:40.831548 6497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:20:40.832244 6512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:40.832415 6497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:20:40.832513 6497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "a3625d8c60a340faabc944133b680bb9"
format_stamp: "Formatted at 2026-09-24 21:20:40 on dist-test-slave-cxpd"
server_key: "f103fd46a3d359b3b6e0ad57dd9081b0"
server_key_iv: "378bbb285de14c8bfd76f514767dfc41"
server_key_version: "encryptionkey@0"
I20260924 21:20:40.832593 6497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:40.841425 6497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:40.843636 6497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:40.843794 6497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:40.847513 6497 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:43559
I20260924 21:20:40.847584 6564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:43559 every 8 connection(s)
I20260924 21:20:40.847922 6497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:40.848838 6565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:40.850966 6565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9: Bootstrap starting.
I20260924 21:20:40.851045 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 6497
I20260924 21:20:40.851190 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:40.851392 20943 external_mini_cluster.cc:1428] Setting key db29d76c89f973999cca877df7baab9a
I20260924 21:20:40.851670 6565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:40.852007 6565 log.cc:824] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:40.852759 6565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9: No bootstrap required, opened a new log
I20260924 21:20:40.855075 6565 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3625d8c60a340faabc944133b680bb9" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43559 } }
I20260924 21:20:40.855252 6565 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:40.855281 6565 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3625d8c60a340faabc944133b680bb9, State: Initialized, Role: FOLLOWER
I20260924 21:20:40.855422 6565 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3625d8c60a340faabc944133b680bb9" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43559 } }
I20260924 21:20:40.855494 6565 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:40.855517 6565 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:40.855583 6565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:40.856155 6565 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3625d8c60a340faabc944133b680bb9" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43559 } }
I20260924 21:20:40.856256 6565 leader_election.cc:302] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3625d8c60a340faabc944133b680bb9; no voters:
I20260924 21:20:40.856403 6565 leader_election.cc:288] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:40.856462 6570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:40.856632 6565 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:40.856638 6570 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [term 1 LEADER]: Becoming Leader. State: Replica: a3625d8c60a340faabc944133b680bb9, State: Running, Role: LEADER
I20260924 21:20:40.856755 6570 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3625d8c60a340faabc944133b680bb9" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43559 } }
I20260924 21:20:40.857163 6570 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3625d8c60a340faabc944133b680bb9. Latest consensus state: current_term: 1 leader_uuid: "a3625d8c60a340faabc944133b680bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3625d8c60a340faabc944133b680bb9" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43559 } } }
I20260924 21:20:40.857359 6570 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:40.857170 6571 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3625d8c60a340faabc944133b680bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3625d8c60a340faabc944133b680bb9" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43559 } } }
I20260924 21:20:40.857610 6571 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9 [sys.catalog]: This master's current role is: LEADER
W20260924 21:20:40.858532 6586 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:20:40.858599 6586 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:20:40.858644 6579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:40.858806 6579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 21:20:40 dist-test-slave-cxpd krb5kdc[6484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:40.860350 6579 catalog_manager.cc:1347] Generated new cluster ID: 99f7873846494102b45217ddadf775bf
I20260924 21:20:40.860405 6579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:40.863009 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:40.851920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49046 (local address 127.20.115.254:43559)
0924 21:20:40.852119 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:40.852128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:40.852163 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:20:40.852748 (+ 585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:40.852756 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:40.852841 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:40.852984 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:40.853592 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:40.855070 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:40.856012 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:40.856647 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:40.856758 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:40.859631 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:40.859635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:40.859635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:40.859646 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:40.861460 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:40.861838 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:40.861841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:40.861841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:40.861938 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:40.862149 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:40.862150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:40.862150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:40.862386 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:40.862568 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:40.862657 (+ 89us) server_negotiation.cc:299] Negotiation successful
0924 21:20:40.862790 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":83,"threads_started":1}
I20260924 21:20:40.871969 6579 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:40.872488 6579 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:40.877867 6579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9: Generated new TSK 0
I20260924 21:20:40.878077 6579 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:20:40.903976 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43559
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:40.984418 6591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:40.984643 6591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:40.984683 6591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:40.985947 6591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:40.986011 6591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:40.986069 6591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:20:40.987581 6591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:40.987844 6591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:40.988075 6591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:20:40.992990 6591 server_base.cc:1034] running on GCE node
W20260924 21:20:40.992944 6597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:40.992954 6596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:40.992954 6599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:40.993397 6591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:40.993618 6591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:40.994794 6591 hybrid_clock.cc:648] HybridClock initialized: now 1790284840994782 us; error 29 us; skew 500 ppm
Sep 24 21:20:40 dist-test-slave-cxpd krb5kdc[6484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:40.997282 6591 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:20:40.997705 6591 webserver.cc:460] Webserver started at http://127.20.115.193:35737/ using document root <none> and password file <none>
I20260924 21:20:40.997947 6591 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:40.998014 6591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:40.998167 6591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:40.999255 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "317d24ad791c40dd8e7824e6dff7082f"
format_stamp: "Formatted at 2026-09-24 21:20:40 on dist-test-slave-cxpd"
server_key: "bca08036694197a260255a8d278a37a6"
server_key_iv: "f45d3fe8b4732bfea6d2834636aa3c38"
server_key_version: "encryptionkey@0"
I20260924 21:20:40.999675 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "317d24ad791c40dd8e7824e6dff7082f"
format_stamp: "Formatted at 2026-09-24 21:20:40 on dist-test-slave-cxpd"
server_key: "bca08036694197a260255a8d278a37a6"
server_key_iv: "f45d3fe8b4732bfea6d2834636aa3c38"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.001160 6591 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:20:41.002099 6606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.002354 6591 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:41.002465 6591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "317d24ad791c40dd8e7824e6dff7082f"
format_stamp: "Formatted at 2026-09-24 21:20:40 on dist-test-slave-cxpd"
server_key: "bca08036694197a260255a8d278a37a6"
server_key_iv: "f45d3fe8b4732bfea6d2834636aa3c38"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.002544 6591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:41.015662 6591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:41.017868 6591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:41.018026 6591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:41.018256 6591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:41.018582 6591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:41.018642 6591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.018678 6591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:41.018692 6591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.024241 6591 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:45907
I20260924 21:20:41.024425 6719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:45907 every 8 connection(s)
I20260924 21:20:41.024657 6591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:20:41.028928 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 6591
I20260924 21:20:41.029067 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:20:41.029325 20943 external_mini_cluster.cc:1428] Setting key 968aaa1c436bbd884a0f70a70da01d8c
Sep 24 21:20:41 dist-test-slave-cxpd krb5kdc[6484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284840, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:41.034260 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.025395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:56093 (local address 127.20.115.254:43559)
0924 21:20:41.025438 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.025440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.025634 (+ 194us) server_negotiation.cc:407] Connection header received
0924 21:20:41.026113 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.026114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.026125 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.026186 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:41.027076 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.027567 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.028518 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.028721 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.028816 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.031238 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:41.031239 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:41.031240 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:41.031248 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:41.032745 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:41.033270 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:41.033271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:41.033271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:41.033332 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:41.033588 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:41.033589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:41.033590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:41.033705 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:41.033777 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.034055 (+ 278us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.034164 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:20:41.034461 6722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.025363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43559 (local address 127.20.115.193:56093)
0924 21:20:41.025520 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.025561 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.026027 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.026261 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.026264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.026809 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:41.026996 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.027001 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.027650 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.027650 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.028442 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.028444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.028829 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.028831 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.028985 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.029542 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:41.029557 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:41.031129 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:41.032905 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:41.032911 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:41.032913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:41.033189 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:41.033400 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:41.033401 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:41.033401 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:41.033512 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:41.033779 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:41.033787 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:41.033926 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.034035 (+ 109us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.034204 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":87,"threads_started":1}
I20260924 21:20:41.034938 6720 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43559
I20260924 21:20:41.035058 6720 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:41.035287 6720 heartbeater.cc:511] Master 127.20.115.254:43559 requested a full tablet report, sending...
I20260924 21:20:41.035847 6529 ts_manager.cc:194] Registered new tserver with Master: 317d24ad791c40dd8e7824e6dff7082f (127.20.115.193:45907)
I20260924 21:20:41.036567 6529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:56093
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:20:41.056844 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43559
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:41.139556 6725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:41.139804 6725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:41.139832 6725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:41.141052 6725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:41.141095 6725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:41.141124 6725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:20:41.142659 6725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:41.142935 6725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:41.143177 6725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:41.148432 6731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:41.148476 6725 server_base.cc:1034] running on GCE node
W20260924 21:20:41.148520 6733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:41.148442 6730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:41.148877 6725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:41.149138 6725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:41.150338 6725 hybrid_clock.cc:648] HybridClock initialized: now 1790284841150324 us; error 29 us; skew 500 ppm
Sep 24 21:20:41 dist-test-slave-cxpd krb5kdc[6484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284841, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:41.153066 6725 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:20:41.153566 6725 webserver.cc:460] Webserver started at http://127.20.115.194:46411/ using document root <none> and password file <none>
I20260924 21:20:41.153817 6725 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:41.153887 6725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:41.154039 6725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:41.155120 6725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "253286272802446aa4bcde9dd584de23"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "9acef90831d1eb2623d490ba25938ad0"
server_key_iv: "32793c70e97e31702232267b5828f8ca"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.155472 6725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "253286272802446aa4bcde9dd584de23"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "9acef90831d1eb2623d490ba25938ad0"
server_key_iv: "32793c70e97e31702232267b5828f8ca"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.156924 6725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260924 21:20:41.157745 6740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.157934 6725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:41.158038 6725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "253286272802446aa4bcde9dd584de23"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "9acef90831d1eb2623d490ba25938ad0"
server_key_iv: "32793c70e97e31702232267b5828f8ca"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.158155 6725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:41.175088 6725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:41.177146 6725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:41.177310 6725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:41.177551 6725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:41.177853 6725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:41.177907 6725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.177951 6725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:41.177977 6725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.184006 6725 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:41649
I20260924 21:20:41.184084 6853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:41649 every 8 connection(s)
I20260924 21:20:41.184415 6725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
Sep 24 21:20:41 dist-test-slave-cxpd krb5kdc[6484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284841, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:41.192203 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 6725
I20260924 21:20:41.192359 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:20:41.192559 20943 external_mini_cluster.cc:1428] Setting key b0e4d3221bfbc10c09feba900fb9a0fa
I20260924 21:20:41.194069 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.185167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:46673 (local address 127.20.115.254:43559)
0924 21:20:41.185229 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.185231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.185409 (+ 178us) server_negotiation.cc:407] Connection header received
0924 21:20:41.185874 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.185875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.185884 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.185931 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:41.186833 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.187340 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.188333 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.188533 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.188598 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.190877 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:41.190878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:41.190878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:41.190885 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:41.192432 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:41.192980 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:41.192981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:41.192981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:41.193038 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:41.193343 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:41.193344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:41.193344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:41.193503 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:41.193611 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.193862 (+ 251us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.193957 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260924 21:20:41.194104 6856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.185141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43559 (local address 127.20.115.194:46673)
0924 21:20:41.185292 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.185328 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.185778 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.185990 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.185993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.186534 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:41.186726 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.186730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.187439 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.187440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.188243 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.188245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.188636 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.188638 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.188748 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.189266 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:41.189274 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:41.190780 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:41.192527 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:41.192535 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:41.192537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:41.192883 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:41.193105 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:41.193106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:41.193106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:41.193245 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:41.193608 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:41.193613 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:41.193726 (+ 113us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.193814 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.193931 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":90,"threads_started":1}
I20260924 21:20:41.195209 6854 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43559
I20260924 21:20:41.195293 6854 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:41.195492 6854 heartbeater.cc:511] Master 127.20.115.254:43559 requested a full tablet report, sending...
I20260924 21:20:41.195873 6529 ts_manager.cc:194] Registered new tserver with Master: 253286272802446aa4bcde9dd584de23 (127.20.115.194:41649)
I20260924 21:20:41.196394 6529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:46673
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:20:41.219448 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43559
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:35117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:41.302198 6859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:41.302402 6859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:41.302424 6859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:41.303570 6859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:41.303611 6859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:41.303637 6859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:20:41.305081 6859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:35117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:41.305266 6859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:41.305457 6859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:41.310491 6867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:41.310496 6865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:41.310611 6859 server_base.cc:1034] running on GCE node
W20260924 21:20:41.310523 6864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:41.310873 6859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:41.311115 6859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:41.312258 6859 hybrid_clock.cc:648] HybridClock initialized: now 1790284841312243 us; error 32 us; skew 500 ppm
Sep 24 21:20:41 dist-test-slave-cxpd krb5kdc[6484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284841, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:41.314781 6859 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:20:41.315224 6859 webserver.cc:460] Webserver started at http://127.20.115.195:33033/ using document root <none> and password file <none>
I20260924 21:20:41.315457 6859 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:41.315524 6859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:41.315675 6859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:41.316674 6859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "8884ac775fb247e1bcde7752349b19f7"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "e9d87219596319d7902f896be3d8766a"
server_key_iv: "983cccaf03735f48261b5881fb3161a3"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.317088 6859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "8884ac775fb247e1bcde7752349b19f7"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "e9d87219596319d7902f896be3d8766a"
server_key_iv: "983cccaf03735f48261b5881fb3161a3"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.318408 6859 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:41.319248 6874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.319453 6859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:41.319552 6859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "8884ac775fb247e1bcde7752349b19f7"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "e9d87219596319d7902f896be3d8766a"
server_key_iv: "983cccaf03735f48261b5881fb3161a3"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.319648 6859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:41.344277 6859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:41.346688 6859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:41.346854 6859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:41.347088 6859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:41.347414 6859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:41.347469 6859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.347512 6859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:41.347538 6859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.353382 6859 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:32819
I20260924 21:20:41.353466 6987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:32819 every 8 connection(s)
I20260924 21:20:41.353820 6859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:20:41.354094 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 6859
I20260924 21:20:41.354221 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:20:41.354439 20943 external_mini_cluster.cc:1428] Setting key c3f25833734933fdba05a341c9f25c40
Sep 24 21:20:41 dist-test-slave-cxpd krb5kdc[6484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284841, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:41.364041 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.354630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:44503 (local address 127.20.115.254:43559)
0924 21:20:41.354676 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.354679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.354931 (+ 252us) server_negotiation.cc:407] Connection header received
0924 21:20:41.355601 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.355602 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.355611 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.355656 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:41.356629 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.357262 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.358602 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.358792 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.358853 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.361011 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:41.361012 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:41.361012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:41.361018 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:41.362584 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:41.363084 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:41.363085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:41.363086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:41.363169 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:41.363503 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:41.363503 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:41.363504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:41.363630 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:41.363683 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.363917 (+ 234us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.363977 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:20:41.364264 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.354559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43559 (local address 127.20.115.195:44503)
0924 21:20:41.354731 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.354823 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.355519 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.355708 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.355711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.356279 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:41.356547 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.356552 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.357351 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.357352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.358512 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.358515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.358874 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.358875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.358978 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.359479 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:41.359487 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:41.360933 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:41.362651 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:41.362657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:41.362658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:41.363011 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:41.363233 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:41.363234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:41.363234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:41.363402 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:41.363692 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:41.363697 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:41.363809 (+ 112us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.363895 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.364012 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260924 21:20:41.364765 6988 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43559
I20260924 21:20:41.364908 6988 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:41.365077 6988 heartbeater.cc:511] Master 127.20.115.254:43559 requested a full tablet report, sending...
I20260924 21:20:41.365433 6529 ts_manager.cc:194] Registered new tserver with Master: 8884ac775fb247e1bcde7752349b19f7 (127.20.115.195:32819)
I20260924 21:20:41.365876 6529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:44503
I20260924 21:20:41.366168 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:20:41.373169 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.367481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.20.115.254:43559)
0924 21:20:41.367519 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.367520 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.367572 (+ 52us) server_negotiation.cc:407] Connection header received
0924 21:20:41.367625 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.367626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.367634 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.367690 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:41.368207 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.368660 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.369442 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.369619 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.369680 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.370895 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:41.370896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:41.370896 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:41.370901 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:41.372038 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:41.372401 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:41.372402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:41.372402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:41.372450 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:41.372625 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:41.372625 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:41.372626 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:41.372740 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:41.372932 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.373030 (+ 98us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.373111 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:20:41.374897 6528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:20:41.376364 6528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:20:41.382936 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.378746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45907 (local address 127.0.0.1:52242)
0924 21:20:41.379032 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.379042 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.379091 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.379440 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.379441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.379443 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:41.379481 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.379483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.380674 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.380675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.381820 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.381822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.382602 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.382603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.382688 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.382701 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.382803 (+ 102us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.382867 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":40,"threads_started":1}
I20260924 21:20:41.382949 7000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.378861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.20.115.193:45907)
0924 21:20:41.379216 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.379219 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.379229 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:20:41.379267 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.379270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.379285 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.379359 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:41.379556 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.380560 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.381898 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.382504 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.382594 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.382653 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.382821 (+ 168us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.382863 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":85,"threads_started":1}
I20260924 21:20:41.382969 6997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.378667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.20.115.194:41649)
0924 21:20:41.378865 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.378870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.378880 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:20:41.378916 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.378918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.378934 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.379013 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:41.379204 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.380304 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.381823 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.382543 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.382614 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.382663 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.382848 (+ 185us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.382888 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":88,"threads_started":1}
I20260924 21:20:41.383270 6996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.378536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:41649 (local address 127.0.0.1:60576)
0924 21:20:41.378667 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.378708 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.378777 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.379023 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.379026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.379028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:41.379143 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.379145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.380425 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.380426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.381728 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.381731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.382697 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.382698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.382762 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.382766 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.382811 (+ 45us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.383205 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":347,"thread_start_us":57,"threads_started":1}
I20260924 21:20:41.384044 6788 tablet_service.cc:1467] Processing CreateTablet for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09 (DEFAULT_TABLE table=test-table [id=5a25066ba66e406488e495c6d1b3972f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:41.384052 6654 tablet_service.cc:1467] Processing CreateTablet for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09 (DEFAULT_TABLE table=test-table [id=5a25066ba66e406488e495c6d1b3972f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:41.384384 6788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:41.384405 6654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:41.384629 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.378880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:32819 (local address 127.0.0.1:56814)
0924 21:20:41.380660 (+ 1780us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.380668 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.380737 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.380922 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.380923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.380925 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:41.380984 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.380985 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.382046 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.382047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.383270 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.383272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.384376 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.384377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.384453 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.384457 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.384523 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.384561 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1769,"thread_start_us":50,"threads_started":1}
I20260924 21:20:41.384924 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.378981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56814 (local address 127.20.115.195:32819)
0924 21:20:41.379151 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.379155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.380736 (+ 1581us) server_negotiation.cc:407] Connection header received
0924 21:20:41.380831 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.380833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.380848 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.380923 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:41.381039 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.381940 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.383528 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.384218 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.384292 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.384339 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.384801 (+ 462us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.384847 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":110,"threads_started":1}
I20260924 21:20:41.385512 6922 tablet_service.cc:1467] Processing CreateTablet for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09 (DEFAULT_TABLE table=test-table [id=5a25066ba66e406488e495c6d1b3972f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:41.385819 6922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:41.387179 7002 tablet_bootstrap.cc:492] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f: Bootstrap starting.
I20260924 21:20:41.387717 7003 tablet_bootstrap.cc:492] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: Bootstrap starting.
I20260924 21:20:41.387837 7002 tablet_bootstrap.cc:654] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:41.388410 7003 tablet_bootstrap.cc:654] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:41.388435 7002 log.cc:824] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:41.389029 7005 tablet_bootstrap.cc:492] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: Bootstrap starting.
I20260924 21:20:41.389030 7003 log.cc:824] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:41.389245 7002 tablet_bootstrap.cc:492] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f: No bootstrap required, opened a new log
I20260924 21:20:41.389307 7002 ts_tablet_manager.cc:1397] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:41.389636 7005 tablet_bootstrap.cc:654] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:41.389782 7003 tablet_bootstrap.cc:492] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: No bootstrap required, opened a new log
I20260924 21:20:41.389847 7003 ts_tablet_manager.cc:1397] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:41.390183 7005 log.cc:824] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:41.390851 7005 tablet_bootstrap.cc:492] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: No bootstrap required, opened a new log
I20260924 21:20:41.390937 7005 ts_tablet_manager.cc:1397] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:20:41.391356 7002 raft_consensus.cc:348] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } }
I20260924 21:20:41.391548 7002 raft_consensus.cc:374] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:41.391588 7002 raft_consensus.cc:729] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 317d24ad791c40dd8e7824e6dff7082f, State: Initialized, Role: FOLLOWER
I20260924 21:20:41.391755 7002 consensus_queue.cc:260] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } }
I20260924 21:20:41.391788 7003 raft_consensus.cc:348] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } }
I20260924 21:20:41.391966 7003 raft_consensus.cc:374] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:41.392000 7003 raft_consensus.cc:729] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 253286272802446aa4bcde9dd584de23, State: Initialized, Role: FOLLOWER
I20260924 21:20:41.392048 7002 ts_tablet_manager.cc:1428] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:20:41.392112 7003 consensus_queue.cc:260] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } }
I20260924 21:20:41.392176 6720 heartbeater.cc:503] Master 127.20.115.254:43559 was elected leader, sending a full tablet report...
I20260924 21:20:41.392402 7003 ts_tablet_manager.cc:1428] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:20:41.392517 6854 heartbeater.cc:503] Master 127.20.115.254:43559 was elected leader, sending a full tablet report...
I20260924 21:20:41.392702 7005 raft_consensus.cc:348] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } }
I20260924 21:20:41.392932 7005 raft_consensus.cc:374] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:41.392984 7005 raft_consensus.cc:729] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8884ac775fb247e1bcde7752349b19f7, State: Initialized, Role: FOLLOWER
I20260924 21:20:41.393100 7005 consensus_queue.cc:260] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } }
I20260924 21:20:41.393399 6988 heartbeater.cc:503] Master 127.20.115.254:43559 was elected leader, sending a full tablet report...
I20260924 21:20:41.393410 7005 ts_tablet_manager.cc:1428] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
W20260924 21:20:41.434988 6855 tablet.cc:2369] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:20:41.468672 7010 raft_consensus.cc:482] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:41.468954 7010 raft_consensus.cc:504] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } }
I20260924 21:20:41.469327 7010 leader_election.cc:288] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 253286272802446aa4bcde9dd584de23 (127.20.115.194:41649), 317d24ad791c40dd8e7824e6dff7082f (127.20.115.193:45907)
I20260924 21:20:41.472302 7000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.469733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:38087 (local address 127.20.115.193:45907)
0924 21:20:41.469787 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.469789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.469923 (+ 134us) server_negotiation.cc:407] Connection header received
0924 21:20:41.469956 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.469957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.469970 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.470019 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:41.470174 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.470641 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.471573 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.471945 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.472034 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.472042 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.472187 (+ 145us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.472222 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260924 21:20:41.472332 6990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.469523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:41649 (local address 127.20.115.195:36441)
0924 21:20:41.469595 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.469611 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.469686 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.469841 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.469842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.469844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:41.469913 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.469915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.470608 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.470609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.471465 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.471468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.472046 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.472047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.472130 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.472133 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.472194 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.472237 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":59}
I20260924 21:20:41.472319 6997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.469630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:36441 (local address 127.20.115.194:41649)
0924 21:20:41.469676 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.469678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.469689 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:20:41.469755 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.469756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.469767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.469822 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:41.470011 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.470496 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.471578 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.471945 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.472032 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.472042 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.472215 (+ 173us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.472250 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:20:41.472350 7011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.469659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45907 (local address 127.20.115.195:38087)
0924 21:20:41.469847 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.469857 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.469907 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.470020 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.470020 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.470022 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:41.470102 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.470103 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.470723 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.470724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.471478 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.471480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.472032 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.472033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.472110 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.472113 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.472180 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.472222 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":126,"threads_started":1}
I20260924 21:20:41.472695 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f31c2120c4d4c1a8c9b3b1662dc6a09" candidate_uuid: "8884ac775fb247e1bcde7752349b19f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "253286272802446aa4bcde9dd584de23" is_pre_election: true
I20260924 21:20:41.472695 6674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f31c2120c4d4c1a8c9b3b1662dc6a09" candidate_uuid: "8884ac775fb247e1bcde7752349b19f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "317d24ad791c40dd8e7824e6dff7082f" is_pre_election: true
I20260924 21:20:41.472870 6808 raft_consensus.cc:2454] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8884ac775fb247e1bcde7752349b19f7 in term 0.
I20260924 21:20:41.472870 6674 raft_consensus.cc:2454] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8884ac775fb247e1bcde7752349b19f7 in term 0.
I20260924 21:20:41.473094 6878 leader_election.cc:302] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 317d24ad791c40dd8e7824e6dff7082f, 8884ac775fb247e1bcde7752349b19f7; no voters:
I20260924 21:20:41.473364 7010 raft_consensus.cc:2781] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:20:41.473433 7010 raft_consensus.cc:482] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:20:41.473459 7010 raft_consensus.cc:3037] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:41.474181 7010 raft_consensus.cc:504] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } }
I20260924 21:20:41.474287 7010 leader_election.cc:288] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [CANDIDATE]: Term 1 election: Requested vote from peers 253286272802446aa4bcde9dd584de23 (127.20.115.194:41649), 317d24ad791c40dd8e7824e6dff7082f (127.20.115.193:45907)
I20260924 21:20:41.474511 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f31c2120c4d4c1a8c9b3b1662dc6a09" candidate_uuid: "8884ac775fb247e1bcde7752349b19f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "253286272802446aa4bcde9dd584de23"
I20260924 21:20:41.474592 6808 raft_consensus.cc:3037] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:41.474565 6674 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f31c2120c4d4c1a8c9b3b1662dc6a09" candidate_uuid: "8884ac775fb247e1bcde7752349b19f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "317d24ad791c40dd8e7824e6dff7082f"
I20260924 21:20:41.474675 6674 raft_consensus.cc:3037] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:41.475356 6808 raft_consensus.cc:2454] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8884ac775fb247e1bcde7752349b19f7 in term 1.
I20260924 21:20:41.475517 6878 leader_election.cc:302] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 253286272802446aa4bcde9dd584de23, 8884ac775fb247e1bcde7752349b19f7; no voters:
I20260924 21:20:41.475566 6674 raft_consensus.cc:2454] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8884ac775fb247e1bcde7752349b19f7 in term 1.
I20260924 21:20:41.475628 7010 raft_consensus.cc:2781] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:41.475791 7010 raft_consensus.cc:686] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 1 LEADER]: Becoming Leader. State: Replica: 8884ac775fb247e1bcde7752349b19f7, State: Running, Role: LEADER
I20260924 21:20:41.475895 7010 consensus_queue.cc:237] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } }
I20260924 21:20:41.476661 6528 catalog_manager.cc:5495] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8884ac775fb247e1bcde7752349b19f7 (127.20.115.195). New cstate: current_term: 1 leader_uuid: "8884ac775fb247e1bcde7752349b19f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8884ac775fb247e1bcde7752349b19f7" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 32819 } health_report { overall_health: HEALTHY } } }
I20260924 21:20:41.500706 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.498372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.20.115.195:32819)
0924 21:20:41.498433 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.498435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.498443 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:20:41.498515 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.498516 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.498532 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.498590 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:20:41.498724 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.499174 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.499928 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.500110 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.500168 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.500332 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:20:41.500414 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:20:41.500483 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.500538 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.500548 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260924 21:20:41.502414 6674 raft_consensus.cc:1261] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [term 1 FOLLOWER]: Refusing update from remote peer 8884ac775fb247e1bcde7752349b19f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:41.502444 6808 raft_consensus.cc:1261] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [term 1 FOLLOWER]: Refusing update from remote peer 8884ac775fb247e1bcde7752349b19f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:41.502763 7010 consensus_queue.cc:1035] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "253286272802446aa4bcde9dd584de23" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:41.502943 7012 consensus_queue.cc:1035] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "317d24ad791c40dd8e7824e6dff7082f" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:41.505316 7016 mvcc.cc:205] Tried to move back new op lower bound from 7333006710791368704 to 7333006710686011392. Current Snapshot: MvccSnapshot[applied={T|T < 7333006710791368704}]
I20260924 21:20:41.506165 7017 mvcc.cc:205] Tried to move back new op lower bound from 7333006710791368704 to 7333006710686011392. Current Snapshot: MvccSnapshot[applied={T|T < 7333006710791368704}]
I20260924 21:20:41.507357 6529 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:41.507452 6529 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:41.508150 6529 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3625d8c60a340faabc944133b680bb9: Sending DeleteTablet for 3 replicas of tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09
I20260924 21:20:41.508459 6788 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:41 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:60576
I20260924 21:20:41.508512 6654 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:41 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:52242
I20260924 21:20:41.508711 7026 tablet_replica.cc:330] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f: stopping tablet replica
I20260924 21:20:41.508752 6922 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:41 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:56814
I20260924 21:20:41.508832 7026 raft_consensus.cc:2229] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:41.508942 7026 raft_consensus.cc:2258] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 317d24ad791c40dd8e7824e6dff7082f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:41.509083 7027 tablet_replica.cc:330] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: stopping tablet replica
I20260924 21:20:41.509168 7027 raft_consensus.cc:2229] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:20:41.509203 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 6591
I20260924 21:20:41.509245 7027 raft_consensus.cc:2258] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:41.509382 7025 tablet_replica.cc:330] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: stopping tablet replica
I20260924 21:20:41.509472 7025 raft_consensus.cc:2229] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:41.509559 7025 raft_consensus.cc:2258] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:41.510142 7027 ts_tablet_manager.cc:1905] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:41.511534 7027 ts_tablet_manager.cc:1918] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:41.511622 7027 log.cc:1196] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/7f31c2120c4d4c1a8c9b3b1662dc6a09
I20260924 21:20:41.511595 7025 ts_tablet_manager.cc:1905] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:41.511878 7027 ts_tablet_manager.cc:1939] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 8884ac775fb247e1bcde7752349b19f7: Deleting consensus metadata
I20260924 21:20:41.512259 6516 catalog_manager.cc:4841] TS 8884ac775fb247e1bcde7752349b19f7 (127.20.115.195:32819): tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09 (table test-table [id=5a25066ba66e406488e495c6d1b3972f]) successfully deleted
I20260924 21:20:41.513572 7025 ts_tablet_manager.cc:1918] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:41.513650 7025 log.cc:1196] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/7f31c2120c4d4c1a8c9b3b1662dc6a09
I20260924 21:20:41.513911 7025 ts_tablet_manager.cc:1939] T 7f31c2120c4d4c1a8c9b3b1662dc6a09 P 253286272802446aa4bcde9dd584de23: Deleting consensus metadata
I20260924 21:20:41.514521 6516 catalog_manager.cc:4841] TS 253286272802446aa4bcde9dd584de23 (127.20.115.194:41649): tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09 (table test-table [id=5a25066ba66e406488e495c6d1b3972f]) successfully deleted
W20260924 21:20:41.515461 6516 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:45907 (error 108)
I20260924 21:20:41.515751 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 6725
I20260924 21:20:41.515913 6999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.515796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45907 (local address 127.0.0.1:52246)
0924 21:20:41.515844 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.515878 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 21:20:41.516348 6516 catalog_manager.cc:4584] TS 317d24ad791c40dd8e7824e6dff7082f (127.20.115.193:45907): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7f31c2120c4d4c1a8c9b3b1662dc6a09: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45907: connect: Connection refused (error 111)
I20260924 21:20:41.521021 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 6859
I20260924 21:20:41.525174 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 6497
2026-09-24T21:20:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (1959 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-24T21:20:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:41Z Disabled control of system clock
I20260924 21:20:41.552428 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43903
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:43903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:20:41.632635 7032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:41.632866 7032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:41.632887 7032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:41.633940 7032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:41.633975 7032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:41.633987 7032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:41.634001 7032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:41.634012 7032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:41.635268 7032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:43903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:41.635444 7032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:41.635675 7032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:41.640446 7040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:41.640503 7037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:41.640460 7038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:41.640504 7032 server_base.cc:1034] running on GCE node
I20260924 21:20:41.640868 7032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:41.641166 7032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:41.642340 7032 hybrid_clock.cc:648] HybridClock initialized: now 1790284841642327 us; error 28 us; skew 500 ppm
I20260924 21:20:41.643592 7032 webserver.cc:460] Webserver started at http://127.20.115.254:39511/ using document root <none> and password file <none>
I20260924 21:20:41.643846 7032 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:41.643893 7032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:41.644012 7032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:41.645161 7032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "791c80d881ea44cc8c35ced8963fb004"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "aaf140e9f041ecbc12bb4465189e7f3a"
server_key_iv: "f19ffa58a4b59cc21731a9b03e1d48f6"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.645509 7032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "791c80d881ea44cc8c35ced8963fb004"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "aaf140e9f041ecbc12bb4465189e7f3a"
server_key_iv: "f19ffa58a4b59cc21731a9b03e1d48f6"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.646745 7032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:41.647491 7046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.647675 7032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:41.647759 7032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "791c80d881ea44cc8c35ced8963fb004"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "aaf140e9f041ecbc12bb4465189e7f3a"
server_key_iv: "f19ffa58a4b59cc21731a9b03e1d48f6"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.647831 7032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:41.668641 7032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:41.668968 7032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:41.669121 7032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:41.673106 7032 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:43903
I20260924 21:20:41.673193 7098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:43903 every 8 connection(s)
I20260924 21:20:41.673501 7032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:41.674521 7099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:41.676658 7099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004: Bootstrap starting.
I20260924 21:20:41.676951 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 7032
I20260924 21:20:41.677091 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:41.677289 20943 external_mini_cluster.cc:1428] Setting key 80db6ac3da6bc69638916e4f32b45510
I20260924 21:20:41.677413 7099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:41.677745 7099 log.cc:824] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:41.678537 7099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004: No bootstrap required, opened a new log
I20260924 21:20:41.680621 7099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c80d881ea44cc8c35ced8963fb004" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43903 } }
I20260924 21:20:41.680799 7099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:41.680835 7099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791c80d881ea44cc8c35ced8963fb004, State: Initialized, Role: FOLLOWER
I20260924 21:20:41.680961 7099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c80d881ea44cc8c35ced8963fb004" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43903 } }
I20260924 21:20:41.681048 7099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:41.681082 7099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:41.681131 7099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:41.681691 7099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c80d881ea44cc8c35ced8963fb004" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43903 } }
I20260924 21:20:41.681810 7099 leader_election.cc:302] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 791c80d881ea44cc8c35ced8963fb004; no voters:
I20260924 21:20:41.681988 7099 leader_election.cc:288] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:41.682025 7104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:41.682147 7104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [term 1 LEADER]: Becoming Leader. State: Replica: 791c80d881ea44cc8c35ced8963fb004, State: Running, Role: LEADER
I20260924 21:20:41.682269 7099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:41.682520 7104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c80d881ea44cc8c35ced8963fb004" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43903 } }
I20260924 21:20:41.682879 7106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "791c80d881ea44cc8c35ced8963fb004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c80d881ea44cc8c35ced8963fb004" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43903 } } }
I20260924 21:20:41.682979 7106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:41.683115 7105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 791c80d881ea44cc8c35ced8963fb004. Latest consensus state: current_term: 1 leader_uuid: "791c80d881ea44cc8c35ced8963fb004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791c80d881ea44cc8c35ced8963fb004" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43903 } } }
I20260924 21:20:41.683281 7105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:41.683357 7113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:41.683517 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.677829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.20.115.254:43903)
0924 21:20:41.678054 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.678064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.678099 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:20:41.678553 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.678560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.678661 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.678775 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:41.678991 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.679862 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.681019 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.681519 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.681608 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.682489 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:41.682492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:41.682493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:41.682713 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:41.682778 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:41.682794 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:41.682858 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.683132 (+ 274us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.683234 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":79,"threads_started":1}
I20260924 21:20:41.683904 7113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:41.685371 7113 catalog_manager.cc:1347] Generated new cluster ID: 6cb1ae0ab3594562a64123d31b8a4d1c
I20260924 21:20:41.685424 7113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:41.700502 7113 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:41.701179 7113 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:41.711508 7113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004: Generated new TSK 0
I20260924 21:20:41.711747 7113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:41.721131 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43903
--builtin_ntp_servers=127.20.115.212:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:20:41.810099 7123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:41.810348 7123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:41.810384 7123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:41.811812 7123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:41.811902 7123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:41.811951 7123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:20:41.813517 7123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:41.813795 7123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:41.814112 7123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:41.820202 7129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:41.820290 7131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:41.820236 7123 server_base.cc:1034] running on GCE node
W20260924 21:20:41.820202 7128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:41.820684 7123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:41.820993 7123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:41.822167 7123 hybrid_clock.cc:648] HybridClock initialized: now 1790284841822155 us; error 30 us; skew 500 ppm
I20260924 21:20:41.823658 7123 webserver.cc:460] Webserver started at http://127.20.115.193:42825/ using document root <none> and password file <none>
I20260924 21:20:41.823927 7123 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:41.823975 7123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:41.824115 7123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:41.825306 7123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "0d8cdfed3d624341b5f3226cb6be5c95"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "ecdaba0cb84870b2b89a6af3448cc03a"
server_key_iv: "a6d1a577d7c7019fe1f911cec75aaf8a"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.825663 7123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "0d8cdfed3d624341b5f3226cb6be5c95"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "ecdaba0cb84870b2b89a6af3448cc03a"
server_key_iv: "a6d1a577d7c7019fe1f911cec75aaf8a"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.826980 7123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:20:41.827740 7137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.827963 7123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:41.828095 7123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "0d8cdfed3d624341b5f3226cb6be5c95"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "ecdaba0cb84870b2b89a6af3448cc03a"
server_key_iv: "a6d1a577d7c7019fe1f911cec75aaf8a"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.828176 7123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:41.859608 7123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:41.859956 7123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:41.860162 7123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:41.860404 7123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:41.860744 7123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:41.860831 7123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.860877 7123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:41.860899 7123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.867172 7123 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:45739
I20260924 21:20:41.867221 7250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:45739 every 8 connection(s)
I20260924 21:20:41.867575 7123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:20:41.872938 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.868408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:44445 (local address 127.20.115.254:43903)
0924 21:20:41.868480 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.868482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.868747 (+ 265us) server_negotiation.cc:407] Connection header received
0924 21:20:41.869244 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.869245 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.869255 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.869320 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:41.869996 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.870496 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.871482 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.871690 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.871750 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.872657 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:41.872658 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:41.872658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:41.872670 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:41.872682 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:41.872685 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:41.872724 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.872836 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.872859 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260924 21:20:41.873159 7253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.868397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43903 (local address 127.20.115.193:44445)
0924 21:20:41.868612 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.868654 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.869155 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.869375 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.869378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.869387 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:41.869894 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.869898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.870605 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.870606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.871395 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.871397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.871788 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.871789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.871906 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.872531 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:20:41.872532 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:41.872560 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:20:41.872582 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:41.872726 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:41.872728 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:41.872730 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.872798 (+ 68us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.872920 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260924 21:20:41.873668 7251 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43903
I20260924 21:20:41.873775 7251 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:41.873951 7251 heartbeater.cc:511] Master 127.20.115.254:43903 requested a full tablet report, sending...
I20260924 21:20:41.874480 7063 ts_manager.cc:194] Registered new tserver with Master: 0d8cdfed3d624341b5f3226cb6be5c95 (127.20.115.193:45739)
I20260924 21:20:41.875120 7063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.193:44445
I20260924 21:20:41.876971 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 7123
I20260924 21:20:41.877108 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:20:41.877318 20943 external_mini_cluster.cc:1428] Setting key c6f0902692625a9892b040d96ea6ea10
I20260924 21:20:41.878213 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43903
--builtin_ntp_servers=127.20.115.212:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:20:41.958693 7254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:41.958904 7254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:41.958925 7254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:41.960052 7254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:41.960100 7254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:41.960125 7254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:20:41.961458 7254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:41.961699 7254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:41.961983 7254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:41.966940 7259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:41.966933 7262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:41.967097 7260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:41.967264 7254 server_base.cc:1034] running on GCE node
I20260924 21:20:41.967478 7254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:41.967702 7254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:41.968847 7254 hybrid_clock.cc:648] HybridClock initialized: now 1790284841968833 us; error 27 us; skew 500 ppm
I20260924 21:20:41.970048 7254 webserver.cc:460] Webserver started at http://127.20.115.194:45325/ using document root <none> and password file <none>
I20260924 21:20:41.970312 7254 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:41.970381 7254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:41.970511 7254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:41.971511 7254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "520ca5cd5f564e24875b030ea95ea130"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "3e0b77a14eccc4b38df735cedea9abe6"
server_key_iv: "1e1efcc4f123a23e1a59031c62c8ba5b"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.971855 7254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "520ca5cd5f564e24875b030ea95ea130"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "3e0b77a14eccc4b38df735cedea9abe6"
server_key_iv: "1e1efcc4f123a23e1a59031c62c8ba5b"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.973177 7254 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:20:41.973857 7268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.974033 7254 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:41.974131 7254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "520ca5cd5f564e24875b030ea95ea130"
format_stamp: "Formatted at 2026-09-24 21:20:41 on dist-test-slave-cxpd"
server_key: "3e0b77a14eccc4b38df735cedea9abe6"
server_key_iv: "1e1efcc4f123a23e1a59031c62c8ba5b"
server_key_version: "encryptionkey@0"
I20260924 21:20:41.974207 7254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:41.986261 7254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:41.986542 7254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:41.986685 7254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:41.986898 7254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:41.987206 7254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:41.987258 7254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.987298 7254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:41.987325 7254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:41.992947 7254 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:41071
I20260924 21:20:41.993010 7381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:41071 every 8 connection(s)
I20260924 21:20:41.993319 7254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:20:41.998479 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.994022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:34351 (local address 127.20.115.254:43903)
0924 21:20:41.994087 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:41.994089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:41.994318 (+ 229us) server_negotiation.cc:407] Connection header received
0924 21:20:41.994777 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:41.994778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:41.994787 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:41.994835 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:41.995531 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.996053 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.997143 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.997334 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.997392 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.998258 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:41.998259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:41.998259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:41.998263 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:41.998282 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:41.998286 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:41.998332 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:41.998397 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 21:20:41.998418 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260924 21:20:41.998704 7384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:41.994015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43903 (local address 127.20.115.194:34351)
0924 21:20:41.994201 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:41.994235 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:41.994697 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:41.994893 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:41.994896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:41.994904 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:41.995431 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.995436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.996194 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.996195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.997065 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:41.997068 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:41.997415 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:41.997416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:41.997518 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:41.998093 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:20:41.998094 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:41.998121 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:20:41.998142 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:41.998333 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:41.998336 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:41.998337 (+ 1us) client_negotiation.cc:769] Sending connection context
0924 21:20:41.998384 (+ 47us) client_negotiation.cc:240] Negotiation successful
0924 21:20:41.998457 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260924 21:20:41.999254 7382 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43903
I20260924 21:20:41.999379 7382 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:41.999584 7382 heartbeater.cc:511] Master 127.20.115.254:43903 requested a full tablet report, sending...
I20260924 21:20:41.999995 7063 ts_manager.cc:194] Registered new tserver with Master: 520ca5cd5f564e24875b030ea95ea130 (127.20.115.194:41071)
I20260924 21:20:42.000394 7063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.194:34351
I20260924 21:20:42.002857 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 7254
I20260924 21:20:42.002988 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:20:42.003221 20943 external_mini_cluster.cc:1428] Setting key 14215d8b64e6ee99a7dd1fe4f48381cc
I20260924 21:20:42.004158 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43903
--builtin_ntp_servers=127.20.115.212:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:20:42.083227 7385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:42.083452 7385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:42.083473 7385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:42.084722 7385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:42.084764 7385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:42.084806 7385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:20:42.086138 7385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:43903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:42.086381 7385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:42.086674 7385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:42.091861 7391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:42.091851 7393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:42.091854 7390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:42.092296 7385 server_base.cc:1034] running on GCE node
I20260924 21:20:42.092504 7385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:42.092708 7385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:42.093873 7385 hybrid_clock.cc:648] HybridClock initialized: now 1790284842093850 us; error 41 us; skew 500 ppm
I20260924 21:20:42.100087 7385 webserver.cc:460] Webserver started at http://127.20.115.195:43203/ using document root <none> and password file <none>
I20260924 21:20:42.100317 7385 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:42.100360 7385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:42.100454 7385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:42.101619 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "8386666181f24a639c907d6937a01900"
format_stamp: "Formatted at 2026-09-24 21:20:42 on dist-test-slave-cxpd"
server_key: "50bad502e87524332d5084fc522b2cd1"
server_key_iv: "1f521cd053ca17c4632860c3b0b33ec4"
server_key_version: "encryptionkey@0"
I20260924 21:20:42.101920 7385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "8386666181f24a639c907d6937a01900"
format_stamp: "Formatted at 2026-09-24 21:20:42 on dist-test-slave-cxpd"
server_key: "50bad502e87524332d5084fc522b2cd1"
server_key_iv: "1f521cd053ca17c4632860c3b0b33ec4"
server_key_version: "encryptionkey@0"
I20260924 21:20:42.103204 7385 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:20:42.104143 7399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:42.104399 7385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:42.104511 7385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "8386666181f24a639c907d6937a01900"
format_stamp: "Formatted at 2026-09-24 21:20:42 on dist-test-slave-cxpd"
server_key: "50bad502e87524332d5084fc522b2cd1"
server_key_iv: "1f521cd053ca17c4632860c3b0b33ec4"
server_key_version: "encryptionkey@0"
I20260924 21:20:42.104605 7385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:42.124761 7385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:42.125135 7385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:42.125244 7385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:42.125428 7385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:42.125785 7385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:42.125847 7385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:42.125892 7385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:42.125917 7385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:42.131707 7385 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:43619
I20260924 21:20:42.131755 7512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:43619 every 8 connection(s)
I20260924 21:20:42.132095 7385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:20:42.137672 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.132895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:46159 (local address 127.20.115.254:43903)
0924 21:20:42.132970 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:42.132973 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:42.133307 (+ 334us) server_negotiation.cc:407] Connection header received
0924 21:20:42.133815 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:42.133817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:42.133827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:42.133887 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:42.134567 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.135072 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.136107 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.136299 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.136363 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.137327 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:42.137328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:42.137328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:42.137332 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:42.137347 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:42.137351 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:42.137407 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:42.137569 (+ 162us) server_negotiation.cc:299] Negotiation successful
0924 21:20:42.137610 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260924 21:20:42.137851 7515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.132873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43903 (local address 127.20.115.195:46159)
0924 21:20:42.133168 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:42.133206 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:42.133689 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:42.133963 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:42.133966 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:42.133974 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:42.134466 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.134469 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.135175 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.135176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.135992 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.135994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.136404 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.136405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.136535 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.137183 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:20:42.137183 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:42.137212 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:20:42.137234 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:42.137454 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:42.137460 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:42.137462 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:20:42.137535 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 21:20:42.137616 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":116,"threads_started":1}
I20260924 21:20:42.138356 7513 heartbeater.cc:347] Connected to a master server at 127.20.115.254:43903
I20260924 21:20:42.138448 7513 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:42.138589 7513 heartbeater.cc:511] Master 127.20.115.254:43903 requested a full tablet report, sending...
I20260924 21:20:42.138849 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 7385
I20260924 21:20:42.138947 7063 ts_manager.cc:194] Registered new tserver with Master: 8386666181f24a639c907d6937a01900 (127.20.115.195:43619)
I20260924 21:20:42.138963 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:20:42.139176 20943 external_mini_cluster.cc:1428] Setting key 7a90ff28c25f0e19077aaed6780106fb
I20260924 21:20:42.139395 7063 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.195:46159
I20260924 21:20:42.140033 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:20:42.144315 7102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.141442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49560 (local address 127.20.115.254:43903)
0924 21:20:42.141478 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:42.141480 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:42.141609 (+ 129us) server_negotiation.cc:407] Connection header received
0924 21:20:42.141670 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:42.141671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:42.141678 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:42.141726 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:42.141866 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.142313 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.143094 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.143294 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.143349 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.144082 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:42.144083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:42.144083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:42.144087 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:42.144100 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:42.144104 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:42.144185 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:42.144248 (+ 63us) server_negotiation.cc:299] Negotiation successful
0924 21:20:42.144258 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:20:42.146406 7062 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:20:42.148108 7062 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:20:42.154531 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.150587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44202 (local address 127.20.115.195:43619)
0924 21:20:42.150755 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:42.150760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:42.150771 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:20:42.150833 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:42.150835 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:42.150852 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:42.150955 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:42.151136 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.152235 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.153546 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.154129 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.154232 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.154322 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:42.154409 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 21:20:42.154445 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260924 21:20:42.154543 7521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.150344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:41071 (local address 127.0.0.1:44588)
0924 21:20:42.150522 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:42.150559 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:42.150618 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:42.151388 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:42.151389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:42.151391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:42.151433 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.151434 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.152613 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.152615 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.153414 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.153416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.154260 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.154261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.154323 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.154328 (+ 5us) client_negotiation.cc:769] Sending connection context
0924 21:20:42.154368 (+ 40us) client_negotiation.cc:240] Negotiation successful
0924 21:20:42.154423 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":27,"thread_start_us":69,"threads_started":1}
I20260924 21:20:42.154546 7522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.150351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:43619 (local address 127.0.0.1:44202)
0924 21:20:42.150715 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:42.150725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:42.150762 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:42.150976 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:42.150979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:42.150981 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:42.151055 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.151056 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.152315 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.152316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.153461 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.153464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.154233 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.154235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.154308 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.154321 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:20:42.154397 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:20:42.154439 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":4,"thread_start_us":58,"threads_started":1}
I20260924 21:20:42.154958 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.150427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44588 (local address 127.20.115.194:41071)
0924 21:20:42.151220 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:42.151224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:42.151233 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:20:42.151270 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:42.151273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:42.151287 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:42.151498 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:42.151568 (+ 70us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.152537 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.153515 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.154159 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.154730 (+ 571us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.154814 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:42.154839 (+ 25us) server_negotiation.cc:299] Negotiation successful
0924 21:20:42.154872 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":85,"threads_started":1}
I20260924 21:20:42.155014 7526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.150606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34854 (local address 127.20.115.193:45739)
0924 21:20:42.150891 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:42.150896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:42.150905 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:20:42.150943 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:42.150946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:42.150959 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:42.151027 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:42.151177 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.152047 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.153800 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.154514 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.154576 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.154671 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:42.154852 (+ 181us) server_negotiation.cc:299] Negotiation successful
0924 21:20:42.154900 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":89,"threads_started":1}
I20260924 21:20:42.154983 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.150372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45739 (local address 127.0.0.1:34854)
0924 21:20:42.150790 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:42.150799 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:42.150841 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:42.151043 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:42.151044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:42.151045 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:42.151081 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.151082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.152132 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.152133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.153715 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.153717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.154707 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.154708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.154762 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.154766 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:42.154824 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 21:20:42.154862 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":37,"thread_start_us":66,"threads_started":1}
I20260924 21:20:42.155689 7316 tablet_service.cc:1467] Processing CreateTablet for tablet 0dd539deecdd4672b17c1d8a7da38c68 (DEFAULT_TABLE table=test-table [id=eb7e1a2f955146b99a7b2608ae20b800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:42.156064 7316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dd539deecdd4672b17c1d8a7da38c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:42.155983 7447 tablet_service.cc:1467] Processing CreateTablet for tablet 0dd539deecdd4672b17c1d8a7da38c68 (DEFAULT_TABLE table=test-table [id=eb7e1a2f955146b99a7b2608ae20b800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:42.156314 7447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dd539deecdd4672b17c1d8a7da38c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:42.156232 7185 tablet_service.cc:1467] Processing CreateTablet for tablet 0dd539deecdd4672b17c1d8a7da38c68 (DEFAULT_TABLE table=test-table [id=eb7e1a2f955146b99a7b2608ae20b800]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:42.156535 7185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dd539deecdd4672b17c1d8a7da38c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:42.159135 7527 tablet_bootstrap.cc:492] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: Bootstrap starting.
I20260924 21:20:42.159688 7528 tablet_bootstrap.cc:492] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95: Bootstrap starting.
I20260924 21:20:42.159816 7527 tablet_bootstrap.cc:654] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:42.159902 7529 tablet_bootstrap.cc:492] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: Bootstrap starting.
I20260924 21:20:42.160189 7527 log.cc:824] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:42.160403 7528 tablet_bootstrap.cc:654] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:42.160511 7529 tablet_bootstrap.cc:654] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:42.160738 7528 log.cc:824] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:42.160861 7529 log.cc:824] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:42.160974 7527 tablet_bootstrap.cc:492] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: No bootstrap required, opened a new log
I20260924 21:20:42.161089 7527 ts_tablet_manager.cc:1397] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:42.161506 7528 tablet_bootstrap.cc:492] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95: No bootstrap required, opened a new log
I20260924 21:20:42.161579 7528 ts_tablet_manager.cc:1397] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260924 21:20:42.163848 7527 raft_consensus.cc:348] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } }
I20260924 21:20:42.164036 7527 raft_consensus.cc:374] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:42.164073 7527 raft_consensus.cc:729] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520ca5cd5f564e24875b030ea95ea130, State: Initialized, Role: FOLLOWER
I20260924 21:20:42.164204 7527 consensus_queue.cc:260] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } }
I20260924 21:20:42.164150 7528 raft_consensus.cc:348] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } }
I20260924 21:20:42.164286 7528 raft_consensus.cc:374] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:42.164321 7528 raft_consensus.cc:729] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d8cdfed3d624341b5f3226cb6be5c95, State: Initialized, Role: FOLLOWER
I20260924 21:20:42.164474 7527 ts_tablet_manager.cc:1428] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:20:42.164441 7528 consensus_queue.cc:260] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } }
I20260924 21:20:42.164623 7382 heartbeater.cc:503] Master 127.20.115.254:43903 was elected leader, sending a full tablet report...
I20260924 21:20:42.164731 7251 heartbeater.cc:503] Master 127.20.115.254:43903 was elected leader, sending a full tablet report...
I20260924 21:20:42.165262 7529 tablet_bootstrap.cc:492] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: No bootstrap required, opened a new log
I20260924 21:20:42.165318 7529 ts_tablet_manager.cc:1397] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260924 21:20:42.167498 7529 raft_consensus.cc:348] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } }
I20260924 21:20:42.167634 7529 raft_consensus.cc:374] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:42.167665 7529 raft_consensus.cc:729] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8386666181f24a639c907d6937a01900, State: Initialized, Role: FOLLOWER
I20260924 21:20:42.167779 7529 consensus_queue.cc:260] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } }
I20260924 21:20:42.167999 7529 ts_tablet_manager.cc:1428] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260924 21:20:42.168094 7513 heartbeater.cc:503] Master 127.20.115.254:43903 was elected leader, sending a full tablet report...
I20260924 21:20:42.168109 7528 ts_tablet_manager.cc:1428] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95: Time spent starting tablet: real 0.006s user 0.000s sys 0.004s
W20260924 21:20:42.243875 7383 tablet.cc:2369] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:20:42.319909 7535 raft_consensus.cc:482] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:42.320073 7535 raft_consensus.cc:504] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } }
I20260924 21:20:42.320539 7535 leader_election.cc:288] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 520ca5cd5f564e24875b030ea95ea130 (127.20.115.194:41071), 0d8cdfed3d624341b5f3226cb6be5c95 (127.20.115.193:45739)
I20260924 21:20:42.323662 7524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.320888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:41783 (local address 127.20.115.194:41071)
0924 21:20:42.320953 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:42.320955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:42.320969 (+ 14us) server_negotiation.cc:407] Connection header received
0924 21:20:42.320997 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:42.320998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:42.321005 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:42.321081 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:42.321278 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.321954 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.322947 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.323310 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.323398 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.323407 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:42.323555 (+ 148us) server_negotiation.cc:299] Negotiation successful
0924 21:20:42.323591 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:20:42.323662 7515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.320754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:41071 (local address 127.20.115.195:41783)
0924 21:20:42.320835 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:42.320848 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:42.320899 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:42.321105 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:42.321107 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:42.321109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:42.321180 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.321182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.322030 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.322031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.322848 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.322851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.323395 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.323396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.323468 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.323471 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:42.323547 (+ 76us) client_negotiation.cc:240] Negotiation successful
0924 21:20:42.323591 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260924 21:20:42.323999 7336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd539deecdd4672b17c1d8a7da38c68" candidate_uuid: "8386666181f24a639c907d6937a01900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "520ca5cd5f564e24875b030ea95ea130" is_pre_election: true
I20260924 21:20:42.324167 7336 raft_consensus.cc:2454] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8386666181f24a639c907d6937a01900 in term 0.
I20260924 21:20:42.324407 7400 leader_election.cc:302] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520ca5cd5f564e24875b030ea95ea130, 8386666181f24a639c907d6937a01900; no voters:
I20260924 21:20:42.324589 7526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.320853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:44485 (local address 127.20.115.193:45739)
0924 21:20:42.320900 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:42.320902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:42.321068 (+ 166us) server_negotiation.cc:407] Connection header received
0924 21:20:42.321731 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:42.321732 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:42.321745 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:42.321775 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:42.321990 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.322703 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.323639 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.324189 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.324269 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.324277 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:42.324472 (+ 195us) server_negotiation.cc:299] Negotiation successful
0924 21:20:42.324515 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:20:42.324564 7535 raft_consensus.cc:2781] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:20:42.324630 7535 raft_consensus.cc:482] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:20:42.324640 7536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.320805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45739 (local address 127.20.115.195:44485)
0924 21:20:42.320982 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:42.320992 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:42.321627 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:42.321839 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:42.321840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:42.321841 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:42.321907 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.321909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.322801 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.322802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.323546 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:42.323548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.324302 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.324303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:42.324387 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.324391 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:42.324446 (+ 55us) client_negotiation.cc:240] Negotiation successful
0924 21:20:42.324513 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":94,"threads_started":1}
I20260924 21:20:42.324673 7535 raft_consensus.cc:3037] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:42.324955 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd539deecdd4672b17c1d8a7da38c68" candidate_uuid: "8386666181f24a639c907d6937a01900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" is_pre_election: true
I20260924 21:20:42.325124 7205 raft_consensus.cc:2454] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8386666181f24a639c907d6937a01900 in term 0.
I20260924 21:20:42.325812 7535 raft_consensus.cc:504] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } }
I20260924 21:20:42.325955 7535 leader_election.cc:288] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [CANDIDATE]: Term 1 election: Requested vote from peers 520ca5cd5f564e24875b030ea95ea130 (127.20.115.194:41071), 0d8cdfed3d624341b5f3226cb6be5c95 (127.20.115.193:45739)
I20260924 21:20:42.326090 7336 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd539deecdd4672b17c1d8a7da38c68" candidate_uuid: "8386666181f24a639c907d6937a01900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "520ca5cd5f564e24875b030ea95ea130"
I20260924 21:20:42.326113 7205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd539deecdd4672b17c1d8a7da38c68" candidate_uuid: "8386666181f24a639c907d6937a01900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d8cdfed3d624341b5f3226cb6be5c95"
I20260924 21:20:42.326177 7205 raft_consensus.cc:3037] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:42.326177 7336 raft_consensus.cc:3037] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:42.327083 7336 raft_consensus.cc:2454] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8386666181f24a639c907d6937a01900 in term 1.
I20260924 21:20:42.327083 7205 raft_consensus.cc:2454] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8386666181f24a639c907d6937a01900 in term 1.
I20260924 21:20:42.327239 7400 leader_election.cc:302] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520ca5cd5f564e24875b030ea95ea130, 8386666181f24a639c907d6937a01900; no voters:
I20260924 21:20:42.327314 7535 raft_consensus.cc:2781] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:42.327455 7535 raft_consensus.cc:686] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 1 LEADER]: Becoming Leader. State: Replica: 8386666181f24a639c907d6937a01900, State: Running, Role: LEADER
I20260924 21:20:42.327548 7535 consensus_queue.cc:237] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } }
I20260924 21:20:42.328375 7063 catalog_manager.cc:5495] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8386666181f24a639c907d6937a01900 (127.20.115.195). New cstate: current_term: 1 leader_uuid: "8386666181f24a639c907d6937a01900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8386666181f24a639c907d6937a01900" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 43619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 } health_report { overall_health: UNKNOWN } } }
I20260924 21:20:42.336416 7525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.333832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44208 (local address 127.20.115.195:43619)
0924 21:20:42.333896 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:42.333898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:42.333907 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:20:42.333942 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:42.333942 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:42.333967 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:42.334044 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:20:42.334185 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.334661 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.335583 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:42.335793 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:42.335871 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:42.336034 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:20:42.336131 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:20:42.336252 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:42.336307 (+ 55us) server_negotiation.cc:299] Negotiation successful
0924 21:20:42.336316 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:20:42.338349 7336 raft_consensus.cc:1261] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [term 1 FOLLOWER]: Refusing update from remote peer 8386666181f24a639c907d6937a01900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:42.338349 7205 raft_consensus.cc:1261] T 0dd539deecdd4672b17c1d8a7da38c68 P 0d8cdfed3d624341b5f3226cb6be5c95 [term 1 FOLLOWER]: Refusing update from remote peer 8386666181f24a639c907d6937a01900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:42.338769 7537 consensus_queue.cc:1035] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "520ca5cd5f564e24875b030ea95ea130" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 41071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:42.338924 7535 consensus_queue.cc:1035] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d8cdfed3d624341b5f3226cb6be5c95" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 45739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:42.341250 7540 mvcc.cc:205] Tried to move back new op lower bound from 7333006714214879232 to 7333006714174267392. Current Snapshot: MvccSnapshot[applied={T|T < 7333006714214879232}]
I20260924 21:20:42.341630 7541 mvcc.cc:205] Tried to move back new op lower bound from 7333006714214879232 to 7333006714174267392. Current Snapshot: MvccSnapshot[applied={T|T < 7333006714214879232}]
I20260924 21:20:42.343760 7063 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:42.343842 7063 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:42.344620 7063 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 791c80d881ea44cc8c35ced8963fb004: Sending DeleteTablet for 3 replicas of tablet 0dd539deecdd4672b17c1d8a7da38c68
I20260924 21:20:42.344942 7447 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dd539deecdd4672b17c1d8a7da38c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:42 UTC) from {username='slave'} at 127.0.0.1:44202
I20260924 21:20:42.345070 7316 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dd539deecdd4672b17c1d8a7da38c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:42 UTC) from {username='slave'} at 127.0.0.1:44588
I20260924 21:20:42.345178 7549 tablet_replica.cc:330] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: stopping tablet replica
I20260924 21:20:42.345268 7550 tablet_replica.cc:330] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: stopping tablet replica
I20260924 21:20:42.345273 7549 raft_consensus.cc:2229] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 1 LEADER]: Raft consensus shutting down.
I20260924 21:20:42.345242 7185 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dd539deecdd4672b17c1d8a7da38c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:42 UTC) from {username='slave'} at 127.0.0.1:34854
I20260924 21:20:42.345357 7550 raft_consensus.cc:2229] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:42.345358 7549 raft_consensus.cc:2258] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:42.345456 7550 raft_consensus.cc:2258] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:42.345863 7550 ts_tablet_manager.cc:1905] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:42.346117 7549 ts_tablet_manager.cc:1905] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:42.346190 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 7123
I20260924 21:20:42.348181 7550 ts_tablet_manager.cc:1918] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:42.348292 7550 log.cc:1196] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/0dd539deecdd4672b17c1d8a7da38c68
I20260924 21:20:42.348609 7550 ts_tablet_manager.cc:1939] T 0dd539deecdd4672b17c1d8a7da38c68 P 520ca5cd5f564e24875b030ea95ea130: Deleting consensus metadata
I20260924 21:20:42.349080 7047 catalog_manager.cc:4841] TS 520ca5cd5f564e24875b030ea95ea130 (127.20.115.194:41071): tablet 0dd539deecdd4672b17c1d8a7da38c68 (table test-table [id=eb7e1a2f955146b99a7b2608ae20b800]) successfully deleted
I20260924 21:20:42.349265 7549 ts_tablet_manager.cc:1918] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:42.349344 7549 log.cc:1196] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/0dd539deecdd4672b17c1d8a7da38c68
I20260924 21:20:42.349627 7549 ts_tablet_manager.cc:1939] T 0dd539deecdd4672b17c1d8a7da38c68 P 8386666181f24a639c907d6937a01900: Deleting consensus metadata
I20260924 21:20:42.349999 7048 catalog_manager.cc:4841] TS 8386666181f24a639c907d6937a01900 (127.20.115.195:43619): tablet 0dd539deecdd4672b17c1d8a7da38c68 (table test-table [id=eb7e1a2f955146b99a7b2608ae20b800]) successfully deleted
W20260924 21:20:42.352174 7049 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:45739 (error 108)
I20260924 21:20:42.352356 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 7254
I20260924 21:20:42.357239 7523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:42.357088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:45739 (local address 127.0.0.1:58456)
0924 21:20:42.357147 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:42.357204 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 21:20:42.357403 7049 catalog_manager.cc:4584] TS 0d8cdfed3d624341b5f3226cb6be5c95 (127.20.115.193:45739): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0dd539deecdd4672b17c1d8a7da38c68: Network error: Client connection negotiation failed: client connection to 127.20.115.193:45739: connect: Connection refused (error 111)
I20260924 21:20:42.359350 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 7385
I20260924 21:20:42.363281 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 7032
2026-09-24T21:20:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (836 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:42 dist-test-slave-cxpd krb5kdc[7555](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:42 dist-test-slave-cxpd krb5kdc[7555](info): set up 2 sockets
Sep 24 21:20:42 dist-test-slave-cxpd krb5kdc[7555](info): commencing operation
krb5kdc: starting...
W20260924 21:20:43.392573 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:43 dist-test-slave-cxpd krb5kdc[7555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:20:43.512100 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:39151
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:38257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:39151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:43.593778 7568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:43.594027 7568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:43.594065 7568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:43.595252 7568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:43.595320 7568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:43.595345 7568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:43.595376 7568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:43.595402 7568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:43.596942 7568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:38257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:39151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:39151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:43.597216 7568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:43.597452 7568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:43.602439 7574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:43.602399 7576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:43.602392 7573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:43.602715 7568 server_base.cc:1034] running on GCE node
I20260924 21:20:43.602957 7568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:43.603252 7568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:43.604421 7568 hybrid_clock.cc:648] HybridClock initialized: now 1790284843604407 us; error 30 us; skew 500 ppm
Sep 24 21:20:43 dist-test-slave-cxpd krb5kdc[7555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284843, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:43.607201 7568 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:20:43.607683 7568 webserver.cc:460] Webserver started at http://127.20.115.254:39103/ using document root <none> and password file <none>
I20260924 21:20:43.607914 7568 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:43.607982 7568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:43.608135 7568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:43.609306 7568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "288854b8718748f9b83cfc4c06f23c69"
format_stamp: "Formatted at 2026-09-24 21:20:43 on dist-test-slave-cxpd"
server_key: "8e7dd9915628b41526fa73c5de58e1e4"
server_key_iv: "80b468dd540dc6c346b0fef47b3593df"
server_key_version: "encryptionkey@0"
I20260924 21:20:43.609674 7568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "288854b8718748f9b83cfc4c06f23c69"
format_stamp: "Formatted at 2026-09-24 21:20:43 on dist-test-slave-cxpd"
server_key: "8e7dd9915628b41526fa73c5de58e1e4"
server_key_iv: "80b468dd540dc6c346b0fef47b3593df"
server_key_version: "encryptionkey@0"
I20260924 21:20:43.611028 7568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:20:43.611819 7583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:43.612020 7568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:43.612138 7568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "288854b8718748f9b83cfc4c06f23c69"
format_stamp: "Formatted at 2026-09-24 21:20:43 on dist-test-slave-cxpd"
server_key: "8e7dd9915628b41526fa73c5de58e1e4"
server_key_iv: "80b468dd540dc6c346b0fef47b3593df"
server_key_version: "encryptionkey@0"
I20260924 21:20:43.612221 7568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:43.636932 7568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:43.639031 7568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:43.639230 7568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:43.643375 7568 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:39151
I20260924 21:20:43.643412 7635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:39151 every 8 connection(s)
I20260924 21:20:43.643762 7568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:43.644755 7636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:43.647064 7636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69: Bootstrap starting.
I20260924 21:20:43.647301 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 7568
I20260924 21:20:43.647435 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:43.647616 7636 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:43.647663 20943 external_mini_cluster.cc:1428] Setting key a457f3bb7c029e3f0cd059eff472cbce
I20260924 21:20:43.647923 7636 log.cc:824] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:43.648748 7636 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69: No bootstrap required, opened a new log
I20260924 21:20:43.650568 7636 raft_consensus.cc:348] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288854b8718748f9b83cfc4c06f23c69" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39151 } }
I20260924 21:20:43.650727 7636 raft_consensus.cc:374] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:43.650774 7636 raft_consensus.cc:729] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 288854b8718748f9b83cfc4c06f23c69, State: Initialized, Role: FOLLOWER
I20260924 21:20:43.650894 7636 consensus_queue.cc:260] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288854b8718748f9b83cfc4c06f23c69" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39151 } }
I20260924 21:20:43.650962 7636 raft_consensus.cc:388] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:43.650983 7636 raft_consensus.cc:482] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:43.651053 7636 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:43.651579 7636 raft_consensus.cc:504] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288854b8718748f9b83cfc4c06f23c69" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39151 } }
I20260924 21:20:43.651662 7636 leader_election.cc:302] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 288854b8718748f9b83cfc4c06f23c69; no voters:
I20260924 21:20:43.651863 7636 leader_election.cc:288] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:43.651888 7641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:43.652002 7641 raft_consensus.cc:686] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [term 1 LEADER]: Becoming Leader. State: Replica: 288854b8718748f9b83cfc4c06f23c69, State: Running, Role: LEADER
I20260924 21:20:43.652140 7636 sys_catalog.cc:564] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:43.652141 7641 consensus_queue.cc:237] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288854b8718748f9b83cfc4c06f23c69" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39151 } }
I20260924 21:20:43.652621 7643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 288854b8718748f9b83cfc4c06f23c69. Latest consensus state: current_term: 1 leader_uuid: "288854b8718748f9b83cfc4c06f23c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288854b8718748f9b83cfc4c06f23c69" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39151 } } }
I20260924 21:20:43.652807 7643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:43.652498 7642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "288854b8718748f9b83cfc4c06f23c69" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288854b8718748f9b83cfc4c06f23c69" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 39151 } } }
I20260924 21:20:43.652869 7642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:43.653231 7649 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:43.653733 7649 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 24 21:20:43 dist-test-slave-cxpd krb5kdc[7555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:43.655350 7649 catalog_manager.cc:1347] Generated new cluster ID: 78fab78f0321410bb19b0953540efc7f
I20260924 21:20:43.655405 7649 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:43.658288 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:43.648197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.20.115.254:39151)
0924 21:20:43.648368 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:43.648378 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:43.648417 (+ 39us) server_negotiation.cc:407] Connection header received
0924 21:20:43.648968 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:43.648976 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:43.649026 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:43.649172 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:43.649796 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:43.650661 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:43.651557 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:43.652172 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:43.652575 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:43.655218 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:43.655222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:43.655222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:43.655234 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:43.656880 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:43.657266 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:43.657268 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:43.657269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:43.657350 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:43.657530 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:43.657530 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:43.657531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:43.657733 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:43.657826 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:43.657942 (+ 116us) server_negotiation.cc:299] Negotiation successful
0924 21:20:43.658040 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260924 21:20:43.665478 7649 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:43.666074 7649 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:43.668759 7649 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69: Generated new TSK 0
I20260924 21:20:43.668982 7649 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.115.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
Entry for principal kudu/127.20.115.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab.
I20260924 21:20:43.700605 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:39151
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:38257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:43.797623 7662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:43.797858 7662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:43.797894 7662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:43.799055 7662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:43.799124 7662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:43.799168 7662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:20:43.800587 7662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:38257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.193.keytab
--principal=kudu/127.20.115.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:39151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:43.800876 7662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:43.801101 7662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:43.806011 7667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:43.806002 7668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:43.806003 7670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:43.806327 7662 server_base.cc:1034] running on GCE node
I20260924 21:20:43.806527 7662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:43.806782 7662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:43.807960 7662 hybrid_clock.cc:648] HybridClock initialized: now 1790284843807940 us; error 34 us; skew 500 ppm
Sep 24 21:20:43 dist-test-slave-cxpd krb5kdc[7555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284843, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:43.810418 7662 init.cc:377] Logged in from keytab as kudu/127.20.115.193@KRBTEST.COM (short username kudu)
I20260924 21:20:43.810904 7662 webserver.cc:460] Webserver started at http://127.20.115.193:42841/ using document root <none> and password file <none>
I20260924 21:20:43.811151 7662 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:43.811195 7662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:43.811334 7662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:43.812368 7662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "01894f69cdf144908b19bd9f28edbbc5"
format_stamp: "Formatted at 2026-09-24 21:20:43 on dist-test-slave-cxpd"
server_key: "139515b9a19779f396e01d9167fa7806"
server_key_iv: "ca9a51d5c8949d3d4911896c713b8142"
server_key_version: "encryptionkey@0"
I20260924 21:20:43.812723 7662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "01894f69cdf144908b19bd9f28edbbc5"
format_stamp: "Formatted at 2026-09-24 21:20:43 on dist-test-slave-cxpd"
server_key: "139515b9a19779f396e01d9167fa7806"
server_key_iv: "ca9a51d5c8949d3d4911896c713b8142"
server_key_version: "encryptionkey@0"
I20260924 21:20:43.814057 7662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:20:43.814817 7677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:43.814977 7662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:20:43.815075 7662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "01894f69cdf144908b19bd9f28edbbc5"
format_stamp: "Formatted at 2026-09-24 21:20:43 on dist-test-slave-cxpd"
server_key: "139515b9a19779f396e01d9167fa7806"
server_key_iv: "ca9a51d5c8949d3d4911896c713b8142"
server_key_version: "encryptionkey@0"
I20260924 21:20:43.815150 7662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:43.825505 7662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:43.827553 7662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:43.827730 7662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:43.827953 7662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:43.828254 7662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:43.828308 7662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:43.828352 7662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:43.828380 7662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:43.834357 7662 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:33119
I20260924 21:20:43.834426 7790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:33119 every 8 connection(s)
I20260924 21:20:43.834748 7662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:20:43.836364 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 7662
I20260924 21:20:43.836511 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:20:43.836709 20943 external_mini_cluster.cc:1428] Setting key 39bf3f938bbd53d9bcca37bb4dd0522c
Sep 24 21:20:43 dist-test-slave-cxpd krb5kdc[7555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284843, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.193@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.115.194@KRBTEST.COM; defaulting to no policy
I20260924 21:20:43.844970 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:43.835505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:35703 (local address 127.20.115.254:39151)
0924 21:20:43.835554 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:43.835556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:43.835776 (+ 220us) server_negotiation.cc:407] Connection header received
0924 21:20:43.836272 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:43.836273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:43.836287 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:43.836367 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:43.837506 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:43.838283 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:43.839714 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:43.839906 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:43.839985 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:43.842124 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:43.842126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:43.842126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:43.842133 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:43.843541 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:43.843985 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:43.843986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:43.843987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:43.844047 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:43.844314 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:43.844314 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:43.844315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:43.844428 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:43.844502 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:43.844773 (+ 271us) server_negotiation.cc:299] Negotiation successful
0924 21:20:43.844880 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:20:43.845153 7793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:43.835492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39151 (local address 127.20.115.193:35703)
0924 21:20:43.835657 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:43.835695 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:43.836151 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:43.836446 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:43.836450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:43.837207 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:43.837435 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:43.837440 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:43.838371 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:43.838371 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:43.839626 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:43.839629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:43.840006 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:43.840007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:43.840139 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:43.840637 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:43.840645 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:43.842021 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:43.843657 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:43.843665 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:43.843667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:43.843891 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:43.844122 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:43.844123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:43.844123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:43.844237 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:43.844493 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:43.844499 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:43.844638 (+ 139us) client_negotiation.cc:769] Sending connection context
0924 21:20:43.844749 (+ 111us) client_negotiation.cc:240] Negotiation successful
0924 21:20:43.844930 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":81,"threads_started":1}
I20260924 21:20:43.845646 7791 heartbeater.cc:347] Connected to a master server at 127.20.115.254:39151
I20260924 21:20:43.845767 7791 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:43.845943 7791 heartbeater.cc:511] Master 127.20.115.254:39151 requested a full tablet report, sending...
I20260924 21:20:43.846451 7600 ts_manager.cc:194] Registered new tserver with Master: 01894f69cdf144908b19bd9f28edbbc5 (127.20.115.193:33119)
I20260924 21:20:43.847113 7600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.193@KRBTEST.COM'} at 127.20.115.193:35703
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
Entry for principal kudu/127.20.115.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab.
I20260924 21:20:43.863021 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:39151
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:38257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:43.941854 7796 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:43.942077 7796 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:43.942116 7796 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:43.943243 7796 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:43.943308 7796 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:43.943353 7796 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:20:43.944854 7796 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:38257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.194.keytab
--principal=kudu/127.20.115.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:39151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7796
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:43.945102 7796 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:43.945333 7796 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:43.950238 7801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:43.950258 7802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:43.950212 7804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:43.950433 7796 server_base.cc:1034] running on GCE node
I20260924 21:20:43.950623 7796 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:43.950860 7796 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:43.952029 7796 hybrid_clock.cc:648] HybridClock initialized: now 1790284843952010 us; error 36 us; skew 500 ppm
Sep 24 21:20:43 dist-test-slave-cxpd krb5kdc[7555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284843, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:43.954638 7796 init.cc:377] Logged in from keytab as kudu/127.20.115.194@KRBTEST.COM (short username kudu)
I20260924 21:20:43.955122 7796 webserver.cc:460] Webserver started at http://127.20.115.194:46341/ using document root <none> and password file <none>
I20260924 21:20:43.955338 7796 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:43.955382 7796 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:43.955528 7796 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:43.956517 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "705f5df3c72c4a4db90c64fc338a31b4"
format_stamp: "Formatted at 2026-09-24 21:20:43 on dist-test-slave-cxpd"
server_key: "adb7f59debcc855453a2e4c452f83ce7"
server_key_iv: "523d341fee66eea6cab7914249a484b7"
server_key_version: "encryptionkey@0"
I20260924 21:20:43.956938 7796 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "705f5df3c72c4a4db90c64fc338a31b4"
format_stamp: "Formatted at 2026-09-24 21:20:43 on dist-test-slave-cxpd"
server_key: "adb7f59debcc855453a2e4c452f83ce7"
server_key_iv: "523d341fee66eea6cab7914249a484b7"
server_key_version: "encryptionkey@0"
I20260924 21:20:43.958303 7796 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:43.959065 7811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:43.959256 7796 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:43.959345 7796 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "705f5df3c72c4a4db90c64fc338a31b4"
format_stamp: "Formatted at 2026-09-24 21:20:43 on dist-test-slave-cxpd"
server_key: "adb7f59debcc855453a2e4c452f83ce7"
server_key_iv: "523d341fee66eea6cab7914249a484b7"
server_key_version: "encryptionkey@0"
I20260924 21:20:43.959419 7796 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:43.993737 7796 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:43.995576 7796 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:43.995739 7796 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:43.995983 7796 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:43.996296 7796 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:43.996328 7796 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:43.996351 7796 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:43.996399 7796 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:44.002256 7796 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:39439
I20260924 21:20:44.002300 7924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:39439 every 8 connection(s)
I20260924 21:20:44.002595 7796 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:20:44.007808 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 7796
I20260924 21:20:44.007913 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:20:44.008111 20943 external_mini_cluster.cc:1428] Setting key 879ddfb7c1e6af7e7988ceee78d216cd
Sep 24 21:20:44 dist-test-slave-cxpd krb5kdc[7555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284843, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.194@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:44.011905 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.003259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:43213 (local address 127.20.115.254:39151)
0924 21:20:44.003306 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.003308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.003547 (+ 239us) server_negotiation.cc:407] Connection header received
0924 21:20:44.003990 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.003991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.004001 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.004052 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:44.004983 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.005472 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.006423 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.006615 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.006672 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.008888 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:44.008889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:44.008890 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:44.008898 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:44.010504 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:44.010908 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:44.010909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:44.010910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:44.010968 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:44.011267 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:44.011268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:44.011268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:44.011383 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:44.011451 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.011763 (+ 312us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.011840 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:20:44.012171 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.003279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39151 (local address 127.20.115.194:43213)
0924 21:20:44.003430 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:44.003465 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:44.003911 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:44.004102 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:44.004106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:44.004652 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:44.004906 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.004912 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.005550 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.005550 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.006341 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.006343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.006692 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.006693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.006786 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.007205 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:44.007213 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:44.008771 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:44.010591 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:44.010598 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:44.010600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:44.010825 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:44.011042 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:44.011043 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:44.011043 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:44.011175 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:44.011454 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:44.011461 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:44.011617 (+ 156us) client_negotiation.cc:769] Sending connection context
0924 21:20:44.011729 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 21:20:44.011903 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":91,"threads_started":1}
I20260924 21:20:44.012707 7925 heartbeater.cc:347] Connected to a master server at 127.20.115.254:39151
I20260924 21:20:44.012838 7925 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:44.013015 7925 heartbeater.cc:511] Master 127.20.115.254:39151 requested a full tablet report, sending...
I20260924 21:20:44.013423 7600 ts_manager.cc:194] Registered new tserver with Master: 705f5df3c72c4a4db90c64fc338a31b4 (127.20.115.194:39439)
I20260924 21:20:44.013972 7600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.194@KRBTEST.COM'} at 127.20.115.194:43213
WARNING: no policy specified for kudu/127.20.115.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
Entry for principal kudu/127.20.115.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab.
I20260924 21:20:44.034973 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:39151
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:38257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:44.115512 7930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:44.115770 7930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:44.115808 7930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:44.117022 7930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:44.117060 7930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:44.117086 7930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:20:44.118597 7930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:38257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.195.keytab
--principal=kudu/127.20.115.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:39151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:44.118820 7930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:44.119047 7930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:44.123980 7936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:44.123970 7938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:44.123972 7935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:44.124177 7930 server_base.cc:1034] running on GCE node
I20260924 21:20:44.124421 7930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:44.124612 7930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:44.125743 7930 hybrid_clock.cc:648] HybridClock initialized: now 1790284844125732 us; error 26 us; skew 500 ppm
Sep 24 21:20:44 dist-test-slave-cxpd krb5kdc[7555](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284844, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:44.128083 7930 init.cc:377] Logged in from keytab as kudu/127.20.115.195@KRBTEST.COM (short username kudu)
I20260924 21:20:44.128523 7930 webserver.cc:460] Webserver started at http://127.20.115.195:40891/ using document root <none> and password file <none>
I20260924 21:20:44.128695 7930 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:44.128736 7930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:44.128878 7930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:44.129829 7930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "cfd8468739f448c9b20fbdbae9de0c5a"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "f92c97cf9317af3a44df58ad169ffd63"
server_key_iv: "d0f14b23b8724606a02b2980d7e739d9"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.130116 7930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "cfd8468739f448c9b20fbdbae9de0c5a"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "f92c97cf9317af3a44df58ad169ffd63"
server_key_iv: "d0f14b23b8724606a02b2980d7e739d9"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.131361 7930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:20:44.132035 7945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:44.132256 7930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:20:44.132326 7930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "cfd8468739f448c9b20fbdbae9de0c5a"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "f92c97cf9317af3a44df58ad169ffd63"
server_key_iv: "d0f14b23b8724606a02b2980d7e739d9"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.132419 7930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:44.158144 7930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:44.160436 7930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:44.160580 7930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:44.160853 7930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:44.161168 7930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:44.161230 7930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:44.161273 7930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:44.161300 7930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:44.167124 7930 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:42241
I20260924 21:20:44.167164 8058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:42241 every 8 connection(s)
I20260924 21:20:44.167474 7930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:20:44.169649 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 7930
I20260924 21:20:44.169772 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:20:44.169952 20943 external_mini_cluster.cc:1428] Setting key d306bde5b93d85106ef572873cb5d749
Sep 24 21:20:44 dist-test-slave-cxpd krb5kdc[7555](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284844, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.195@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:20:44.176692 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.168162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:38143 (local address 127.20.115.254:39151)
0924 21:20:44.168223 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.168226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.168434 (+ 208us) server_negotiation.cc:407] Connection header received
0924 21:20:44.168989 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.168990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.169000 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.169058 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:44.170003 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.170814 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.171783 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.171974 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.172031 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.174225 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:44.174226 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:44.174227 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:44.174233 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:44.175625 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:44.175958 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:44.175958 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:44.175959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:44.176008 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:44.176222 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:44.176223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:44.176223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:44.176318 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:44.176367 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.176574 (+ 207us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.176629 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260924 21:20:44.176927 8061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.168158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:39151 (local address 127.20.115.195:38143)
0924 21:20:44.168304 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:44.168338 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:44.168877 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:44.169111 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:44.169114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:44.169649 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:44.169908 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.169920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.170903 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.170904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.171703 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.171706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.172051 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.172052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.172146 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.172681 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:20:44.172691 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:44.174134 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:44.175694 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:44.175699 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:44.175700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:44.175896 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:44.176064 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:20:44.176065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:20:44.176065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:20:44.176159 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:20:44.176372 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:44.176375 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:44.176481 (+ 106us) client_negotiation.cc:769] Sending connection context
0924 21:20:44.176554 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 21:20:44.176667 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260924 21:20:44.177428 8059 heartbeater.cc:347] Connected to a master server at 127.20.115.254:39151
I20260924 21:20:44.177537 8059 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:44.177717 8059 heartbeater.cc:511] Master 127.20.115.254:39151 requested a full tablet report, sending...
I20260924 21:20:44.178074 7600 ts_manager.cc:194] Registered new tserver with Master: cfd8468739f448c9b20fbdbae9de0c5a (127.20.115.195:42241)
I20260924 21:20:44.178409 7600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.115.195@KRBTEST.COM'} at 127.20.115.195:38143
I20260924 21:20:44.181656 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:20:44.188864 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.182949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55470 (local address 127.20.115.254:39151)
0924 21:20:44.183024 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.183026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.183137 (+ 111us) server_negotiation.cc:407] Connection header received
0924 21:20:44.183210 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.183211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.183222 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.183296 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:44.183852 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.184314 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.185119 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.185304 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.185390 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.186523 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:44.186524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:44.186525 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:44.186532 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:44.187615 (+ 1083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:44.187965 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:44.187966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:44.187967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:44.188057 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:44.188244 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:44.188246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:44.188246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:44.188407 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:44.188517 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.188655 (+ 138us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.188731 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:20:44.190299 7600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:20:44.191803 7600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:20:44.198257 8072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.194214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:39439 (local address 127.0.0.1:39148)
0924 21:20:44.194343 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:44.194352 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:44.194388 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:44.195126 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:44.195127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:44.195130 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:44.195166 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.195168 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.196340 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.196341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.197264 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.197267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.197948 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.197949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.198048 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.198066 (+ 18us) client_negotiation.cc:769] Sending connection context
0924 21:20:44.198141 (+ 75us) client_negotiation.cc:240] Negotiation successful
0924 21:20:44.198191 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260924 21:20:44.198275 8075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.194756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39148 (local address 127.20.115.194:39439)
0924 21:20:44.194942 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.194946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.194957 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:20:44.194993 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.195007 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.195022 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.195231 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:44.195298 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.196237 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.197353 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.197845 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.197947 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.198006 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.198173 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.198213 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":85,"threads_started":1}
I20260924 21:20:44.199069 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.193991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:33119 (local address 127.0.0.1:59590)
0924 21:20:44.194100 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:44.194140 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:44.194199 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:44.194655 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:44.194658 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:44.194660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:44.194725 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.194727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.196015 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.196016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.197256 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.197258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.198833 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.198834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.198882 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.198885 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:44.198962 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 21:20:44.199003 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":7,"thread_start_us":65,"threads_started":1}
I20260924 21:20:44.199079 7859 tablet_service.cc:1467] Processing CreateTablet for tablet 67aa3a46ad24483b82f471ae8eea8cbd (DEFAULT_TABLE table=test-table [id=358e772bcbb144f5a7f50bfa5236cf6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:44.199102 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.194370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59590 (local address 127.20.115.193:33119)
0924 21:20:44.194503 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.194508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.194517 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:20:44.194547 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.194549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.194562 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.194652 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:44.194785 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.195912 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.198045 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.198746 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.198821 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.198871 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.198981 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.199022 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260924 21:20:44.199460 7859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67aa3a46ad24483b82f471ae8eea8cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:44.199980 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.194074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:42241 (local address 127.0.0.1:38050)
0924 21:20:44.194205 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:44.194215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:44.194268 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:44.194873 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:44.194874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:44.194876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:44.194928 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.194930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.195858 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.195859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.197255 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.197258 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.199815 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.199816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.199865 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.199868 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:44.199906 (+ 38us) client_negotiation.cc:240] Negotiation successful
0924 21:20:44.199936 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":33,"thread_start_us":94,"threads_started":1}
I20260924 21:20:44.200048 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.194503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38050 (local address 127.20.115.195:42241)
0924 21:20:44.194643 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.194647 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.194658 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:20:44.194702 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.194704 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.194719 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.194801 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:44.195028 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.195756 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.198908 (+ 3152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.199734 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.199850 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.199904 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.199930 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.199967 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260924 21:20:44.200752 7993 tablet_service.cc:1467] Processing CreateTablet for tablet 67aa3a46ad24483b82f471ae8eea8cbd (DEFAULT_TABLE table=test-table [id=358e772bcbb144f5a7f50bfa5236cf6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:44.201113 7993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67aa3a46ad24483b82f471ae8eea8cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:44.202925 8076 tablet_bootstrap.cc:492] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: Bootstrap starting.
I20260924 21:20:44.203598 8076 tablet_bootstrap.cc:654] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:44.203780 7725 tablet_service.cc:1467] Processing CreateTablet for tablet 67aa3a46ad24483b82f471ae8eea8cbd (DEFAULT_TABLE table=test-table [id=358e772bcbb144f5a7f50bfa5236cf6a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:44.204115 7725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67aa3a46ad24483b82f471ae8eea8cbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:44.204192 8076 log.cc:824] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:44.204360 8077 tablet_bootstrap.cc:492] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: Bootstrap starting.
I20260924 21:20:44.204998 8077 tablet_bootstrap.cc:654] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:44.205051 8076 tablet_bootstrap.cc:492] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: No bootstrap required, opened a new log
I20260924 21:20:44.205122 8076 ts_tablet_manager.cc:1397] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:44.205673 8077 log.cc:824] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:44.206475 8077 tablet_bootstrap.cc:492] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: No bootstrap required, opened a new log
I20260924 21:20:44.206579 8077 ts_tablet_manager.cc:1397] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:44.207173 8080 tablet_bootstrap.cc:492] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5: Bootstrap starting.
I20260924 21:20:44.207245 8076 raft_consensus.cc:348] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } }
I20260924 21:20:44.207433 8076 raft_consensus.cc:374] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:44.207472 8076 raft_consensus.cc:729] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 705f5df3c72c4a4db90c64fc338a31b4, State: Initialized, Role: FOLLOWER
I20260924 21:20:44.207610 8076 consensus_queue.cc:260] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } }
I20260924 21:20:44.207827 8080 tablet_bootstrap.cc:654] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:44.207890 8076 ts_tablet_manager.cc:1428] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:20:44.208020 7925 heartbeater.cc:503] Master 127.20.115.254:39151 was elected leader, sending a full tablet report...
I20260924 21:20:44.208451 8080 log.cc:824] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:44.208637 8077 raft_consensus.cc:348] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } }
I20260924 21:20:44.208828 8077 raft_consensus.cc:374] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:44.208878 8077 raft_consensus.cc:729] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfd8468739f448c9b20fbdbae9de0c5a, State: Initialized, Role: FOLLOWER
I20260924 21:20:44.208974 8077 consensus_queue.cc:260] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } }
I20260924 21:20:44.209261 8077 ts_tablet_manager.cc:1428] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260924 21:20:44.209295 8080 tablet_bootstrap.cc:492] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5: No bootstrap required, opened a new log
I20260924 21:20:44.209270 8059 heartbeater.cc:503] Master 127.20.115.254:39151 was elected leader, sending a full tablet report...
I20260924 21:20:44.209352 8080 ts_tablet_manager.cc:1397] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:44.210938 8080 raft_consensus.cc:348] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } }
I20260924 21:20:44.211061 8080 raft_consensus.cc:374] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:44.211083 8080 raft_consensus.cc:729] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01894f69cdf144908b19bd9f28edbbc5, State: Initialized, Role: FOLLOWER
I20260924 21:20:44.211184 8080 consensus_queue.cc:260] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } }
I20260924 21:20:44.211429 8080 ts_tablet_manager.cc:1428] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260924 21:20:44.211467 7791 heartbeater.cc:503] Master 127.20.115.254:39151 was elected leader, sending a full tablet report...
W20260924 21:20:44.253247 7926 tablet.cc:2369] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:20:44.335466 7792 tablet.cc:2369] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:20:44.418059 8060 tablet.cc:2369] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:20:44.466233 8084 raft_consensus.cc:482] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:44.466361 8084 raft_consensus.cc:504] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } }
I20260924 21:20:44.466912 8084 leader_election.cc:288] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 705f5df3c72c4a4db90c64fc338a31b4 (127.20.115.194:39439), cfd8468739f448c9b20fbdbae9de0c5a (127.20.115.195:42241)
I20260924 21:20:44.470127 8075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.467201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:51751 (local address 127.20.115.194:39439)
0924 21:20:44.467258 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.467260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.467379 (+ 119us) server_negotiation.cc:407] Connection header received
0924 21:20:44.467419 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.467421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.467434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.467501 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:44.467675 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.468186 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.469394 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.469765 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.469820 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.469828 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.470012 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.470056 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260924 21:20:44.470148 8074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.467283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:52909 (local address 127.20.115.195:42241)
0924 21:20:44.467356 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.467358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.467396 (+ 38us) server_negotiation.cc:407] Connection header received
0924 21:20:44.467505 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.467505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.467515 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.467575 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:44.467760 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.468243 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.469394 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.469783 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.469878 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.469886 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.470047 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.470088 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260924 21:20:44.470153 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.467135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:39439 (local address 127.20.115.193:51751)
0924 21:20:44.467278 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:44.467289 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:44.467362 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:44.467509 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:44.467510 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:44.467513 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:44.467596 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.467598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.468274 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.468275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.469305 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.469307 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.469852 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.469854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.469925 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.469929 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:44.469983 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 21:20:44.470051 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":11,"thread_start_us":68,"threads_started":1}
I20260924 21:20:44.470160 8086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.467198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:42241 (local address 127.20.115.193:52909)
0924 21:20:44.467334 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:44.467346 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:44.467412 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:44.467592 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:44.467593 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:44.467595 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:44.467687 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.467689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.468333 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.468334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.469306 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.469308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.469902 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.469902 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.469955 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.469958 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:44.470031 (+ 73us) client_negotiation.cc:240] Negotiation successful
0924 21:20:44.470069 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":4,"thread_start_us":69,"threads_started":1}
I20260924 21:20:44.470515 7879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67aa3a46ad24483b82f471ae8eea8cbd" candidate_uuid: "01894f69cdf144908b19bd9f28edbbc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "705f5df3c72c4a4db90c64fc338a31b4" is_pre_election: true
I20260924 21:20:44.470679 7879 raft_consensus.cc:2454] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01894f69cdf144908b19bd9f28edbbc5 in term 0.
I20260924 21:20:44.470724 8013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67aa3a46ad24483b82f471ae8eea8cbd" candidate_uuid: "01894f69cdf144908b19bd9f28edbbc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" is_pre_election: true
I20260924 21:20:44.470893 8013 raft_consensus.cc:2454] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01894f69cdf144908b19bd9f28edbbc5 in term 0.
I20260924 21:20:44.470923 7681 leader_election.cc:302] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01894f69cdf144908b19bd9f28edbbc5, 705f5df3c72c4a4db90c64fc338a31b4; no voters:
I20260924 21:20:44.471040 8084 raft_consensus.cc:2781] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:20:44.471131 8084 raft_consensus.cc:482] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:20:44.471164 8084 raft_consensus.cc:3037] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:44.471880 8084 raft_consensus.cc:504] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } }
I20260924 21:20:44.472038 8084 leader_election.cc:288] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [CANDIDATE]: Term 1 election: Requested vote from peers 705f5df3c72c4a4db90c64fc338a31b4 (127.20.115.194:39439), cfd8468739f448c9b20fbdbae9de0c5a (127.20.115.195:42241)
I20260924 21:20:44.472187 7879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67aa3a46ad24483b82f471ae8eea8cbd" candidate_uuid: "01894f69cdf144908b19bd9f28edbbc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "705f5df3c72c4a4db90c64fc338a31b4"
I20260924 21:20:44.472232 8013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67aa3a46ad24483b82f471ae8eea8cbd" candidate_uuid: "01894f69cdf144908b19bd9f28edbbc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfd8468739f448c9b20fbdbae9de0c5a"
I20260924 21:20:44.472277 7879 raft_consensus.cc:3037] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:44.472295 8013 raft_consensus.cc:3037] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:44.473212 8013 raft_consensus.cc:2454] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01894f69cdf144908b19bd9f28edbbc5 in term 1.
I20260924 21:20:44.473212 7879 raft_consensus.cc:2454] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01894f69cdf144908b19bd9f28edbbc5 in term 1.
I20260924 21:20:44.473383 7681 leader_election.cc:302] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01894f69cdf144908b19bd9f28edbbc5, 705f5df3c72c4a4db90c64fc338a31b4; no voters:
I20260924 21:20:44.473479 8084 raft_consensus.cc:2781] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:44.473675 8084 raft_consensus.cc:686] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [term 1 LEADER]: Becoming Leader. State: Replica: 01894f69cdf144908b19bd9f28edbbc5, State: Running, Role: LEADER
I20260924 21:20:44.473780 8084 consensus_queue.cc:237] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } }
I20260924 21:20:44.474653 7599 catalog_manager.cc:5495] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01894f69cdf144908b19bd9f28edbbc5 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "01894f69cdf144908b19bd9f28edbbc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01894f69cdf144908b19bd9f28edbbc5" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 33119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 } health_report { overall_health: UNKNOWN } } }
I20260924 21:20:44.549886 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.547417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59604 (local address 127.20.115.193:33119)
0924 21:20:44.547491 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.547493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.547501 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:20:44.547545 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.547546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.547566 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.547646 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:20:44.547811 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.548294 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.549084 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.549291 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.549378 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.549541 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:20:44.549658 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:20:44.549758 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.549777 (+ 19us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.549797 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260924 21:20:44.551806 7879 raft_consensus.cc:1261] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [term 1 FOLLOWER]: Refusing update from remote peer 01894f69cdf144908b19bd9f28edbbc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:44.551813 8013 raft_consensus.cc:1261] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [term 1 FOLLOWER]: Refusing update from remote peer 01894f69cdf144908b19bd9f28edbbc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:44.552137 8087 consensus_queue.cc:1035] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "705f5df3c72c4a4db90c64fc338a31b4" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 39439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:44.552263 8084 consensus_queue.cc:1035] T 67aa3a46ad24483b82f471ae8eea8cbd P 01894f69cdf144908b19bd9f28edbbc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfd8468739f448c9b20fbdbae9de0c5a" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 42241 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:44.554571 8090 mvcc.cc:205] Tried to move back new op lower bound from 7333006723281317888 to 7333006722965274624. Current Snapshot: MvccSnapshot[applied={T|T < 7333006723281317888}]
I20260924 21:20:44.554757 8091 mvcc.cc:205] Tried to move back new op lower bound from 7333006723281317888 to 7333006722965274624. Current Snapshot: MvccSnapshot[applied={T|T < 7333006723281317888}]
I20260924 21:20:44.555117 8092 mvcc.cc:205] Tried to move back new op lower bound from 7333006723281317888 to 7333006722965274624. Current Snapshot: MvccSnapshot[applied={T|T < 7333006723281317888}]
I20260924 21:20:44.557164 7600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:44.557278 7600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:44.557988 7600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 288854b8718748f9b83cfc4c06f23c69: Sending DeleteTablet for 3 replicas of tablet 67aa3a46ad24483b82f471ae8eea8cbd
I20260924 21:20:44.558334 7725 tablet_service.cc:1514] Processing DeleteTablet for tablet 67aa3a46ad24483b82f471ae8eea8cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:44 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:59590
I20260924 21:20:44.558385 7993 tablet_service.cc:1514] Processing DeleteTablet for tablet 67aa3a46ad24483b82f471ae8eea8cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:44 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:38050
I20260924 21:20:44.558563 7859 tablet_service.cc:1514] Processing DeleteTablet for tablet 67aa3a46ad24483b82f471ae8eea8cbd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:44 UTC) from {username='kudu', principal='kudu/127.20.115.254@KRBTEST.COM'} at 127.0.0.1:39148
I20260924 21:20:44.558581 8101 tablet_replica.cc:330] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: stopping tablet replica
I20260924 21:20:44.558681 8101 raft_consensus.cc:2229] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:44.558763 8102 tablet_replica.cc:330] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: stopping tablet replica
I20260924 21:20:44.558779 8101 raft_consensus.cc:2258] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:44.558868 8102 raft_consensus.cc:2229] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:44.558987 8102 raft_consensus.cc:2258] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:44.559053 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 7662
I20260924 21:20:44.559293 8101 ts_tablet_manager.cc:1905] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:44.560168 8102 ts_tablet_manager.cc:1905] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:44.560935 8101 ts_tablet_manager.cc:1918] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:44.561028 8101 log.cc:1196] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/67aa3a46ad24483b82f471ae8eea8cbd
I20260924 21:20:44.561342 8101 ts_tablet_manager.cc:1939] T 67aa3a46ad24483b82f471ae8eea8cbd P cfd8468739f448c9b20fbdbae9de0c5a: Deleting consensus metadata
I20260924 21:20:44.561780 7586 catalog_manager.cc:4841] TS cfd8468739f448c9b20fbdbae9de0c5a (127.20.115.195:42241): tablet 67aa3a46ad24483b82f471ae8eea8cbd (table test-table [id=358e772bcbb144f5a7f50bfa5236cf6a]) successfully deleted
I20260924 21:20:44.562096 8102 ts_tablet_manager.cc:1918] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:44.562171 8102 log.cc:1196] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/67aa3a46ad24483b82f471ae8eea8cbd
I20260924 21:20:44.562438 8102 ts_tablet_manager.cc:1939] T 67aa3a46ad24483b82f471ae8eea8cbd P 705f5df3c72c4a4db90c64fc338a31b4: Deleting consensus metadata
I20260924 21:20:44.563052 7587 catalog_manager.cc:4841] TS 705f5df3c72c4a4db90c64fc338a31b4 (127.20.115.194:39439): tablet 67aa3a46ad24483b82f471ae8eea8cbd (table test-table [id=358e772bcbb144f5a7f50bfa5236cf6a]) successfully deleted
W20260924 21:20:44.565914 7587 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:33119 (error 108)
I20260924 21:20:44.566471 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.566353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:33119 (local address 127.0.0.1:59610)
0924 21:20:44.566405 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:44.566441 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:33119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260924 21:20:44.566502 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 7796
W20260924 21:20:44.567092 7587 catalog_manager.cc:4584] TS 01894f69cdf144908b19bd9f28edbbc5 (127.20.115.193:33119): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67aa3a46ad24483b82f471ae8eea8cbd: Network error: Client connection negotiation failed: client connection to 127.20.115.193:33119: connect: Connection refused (error 111)
I20260924 21:20:44.572151 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 7930
I20260924 21:20:44.576679 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 7568
2026-09-24T21:20:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2214 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-24T21:20:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:44Z Disabled control of system clock
I20260924 21:20:44.604028 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45001
--webserver_interface=127.20.115.254
--webserver_port=0
--builtin_ntp_servers=127.20.115.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:45001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:20:44.684350 8107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:44.684564 8107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:44.684585 8107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:44.685822 8107 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:44.685864 8107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:44.685878 8107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:44.685913 8107 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:44.685940 8107 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:44.687273 8107 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:45001
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:45001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8107
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:44.687497 8107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:44.687842 8107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:20:44.692690 8107 server_base.cc:1034] running on GCE node
W20260924 21:20:44.692668 8113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:44.692704 8112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:44.692673 8115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:44.693104 8107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:44.693393 8107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:44.694583 8107 hybrid_clock.cc:648] HybridClock initialized: now 1790284844694576 us; error 35 us; skew 500 ppm
I20260924 21:20:44.695775 8107 webserver.cc:460] Webserver started at http://127.20.115.254:37657/ using document root <none> and password file <none>
I20260924 21:20:44.696071 8107 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:44.696120 8107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:44.696280 8107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:44.697489 8107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "33cfec9d81e9ac90eafe658335bb9c0e"
server_key_iv: "0e2fac5a2c3e4ccbb111205dbe327f12"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.697842 8107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "33cfec9d81e9ac90eafe658335bb9c0e"
server_key_iv: "0e2fac5a2c3e4ccbb111205dbe327f12"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.699123 8107 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:20:44.699858 8121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:44.700044 8107 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:44.700143 8107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "33cfec9d81e9ac90eafe658335bb9c0e"
server_key_iv: "0e2fac5a2c3e4ccbb111205dbe327f12"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.700237 8107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:44.722908 8107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:44.723230 8107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:44.723385 8107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:44.727224 8107 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:45001
I20260924 21:20:44.727258 8173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:45001 every 8 connection(s)
I20260924 21:20:44.727589 8107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:44.728417 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 8107
I20260924 21:20:44.728552 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:44.728688 8174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:44.728818 20943 external_mini_cluster.cc:1428] Setting key 19e5c6b7abc386bac0d44fa91f91b624
I20260924 21:20:44.731598 8174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec: Bootstrap starting.
I20260924 21:20:44.732245 8174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:44.732537 8174 log.cc:824] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:44.733228 8174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec: No bootstrap required, opened a new log
I20260924 21:20:44.734402 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.729363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46162 (local address 127.20.115.254:45001)
0924 21:20:44.729565 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.729572 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.729597 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:20:44.730111 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.730118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.730177 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.730293 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:44.730509 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.731357 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.732446 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.732947 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.733088 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.733791 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:44.733795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:44.733796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:44.733963 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:44.733994 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:44.734004 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:44.734081 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.734159 (+ 78us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.734234 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
I20260924 21:20:44.735188 8174 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45001 } }
I20260924 21:20:44.735337 8174 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:44.735380 8174 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04cd43c1d7f74ccea1a9bf4d3d1c5fec, State: Initialized, Role: FOLLOWER
I20260924 21:20:44.735517 8174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45001 } }
I20260924 21:20:44.735594 8174 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:44.735635 8174 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:44.735682 8174 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:44.736171 8174 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45001 } }
I20260924 21:20:44.736284 8174 leader_election.cc:302] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04cd43c1d7f74ccea1a9bf4d3d1c5fec; no voters:
I20260924 21:20:44.736464 8174 leader_election.cc:288] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:44.736507 8179 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:44.736665 8179 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [term 1 LEADER]: Becoming Leader. State: Replica: 04cd43c1d7f74ccea1a9bf4d3d1c5fec, State: Running, Role: LEADER
I20260924 21:20:44.736721 8174 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:44.736874 8179 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45001 } }
I20260924 21:20:44.737251 8180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45001 } } }
I20260924 21:20:44.737443 8181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04cd43c1d7f74ccea1a9bf4d3d1c5fec. Latest consensus state: current_term: 1 leader_uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04cd43c1d7f74ccea1a9bf4d3d1c5fec" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 45001 } } }
I20260924 21:20:44.737530 8180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:44.737552 8181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:44.737725 8188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:44.738196 8188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:44.739730 8188 catalog_manager.cc:1347] Generated new cluster ID: bf2ff8eb058b4cb09995bfa47eaca539
I20260924 21:20:44.739778 8188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:44.760257 8188 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:44.760890 8188 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:44.766218 8188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec: Generated new TSK 0
I20260924 21:20:44.766412 8188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:44.770169 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.193:0
--local_ip_for_outbound_sockets=127.20.115.193
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45001
--builtin_ntp_servers=127.20.115.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:20:44.866633 8198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:44.866837 8198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:44.866858 8198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:44.868010 8198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:44.868049 8198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:44.868074 8198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.193
I20260924 21:20:44.869464 8198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.115.193
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.193
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:44.869716 8198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:44.870013 8198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:44.874845 8204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:44.874909 8203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:44.874951 8206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:44.875178 8198 server_base.cc:1034] running on GCE node
I20260924 21:20:44.875368 8198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:44.875602 8198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:44.876756 8198 hybrid_clock.cc:648] HybridClock initialized: now 1790284844876741 us; error 34 us; skew 500 ppm
I20260924 21:20:44.878103 8198 webserver.cc:460] Webserver started at http://127.20.115.193:41787/ using document root <none> and password file <none>
I20260924 21:20:44.878360 8198 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:44.878417 8198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:44.878535 8198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:44.879926 8198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/data/instance:
uuid: "c80adeaf11d84b08a2f4fb74689e0439"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "658d824af28b0a8fedad6b0ca0743f59"
server_key_iv: "d692a6104d09c3015660c3050895bbf1"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.880314 8198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance:
uuid: "c80adeaf11d84b08a2f4fb74689e0439"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "658d824af28b0a8fedad6b0ca0743f59"
server_key_iv: "d692a6104d09c3015660c3050895bbf1"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.881881 8198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260924 21:20:44.882714 8212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:44.882910 8198 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:20:44.883005 8198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/wal
uuid: "c80adeaf11d84b08a2f4fb74689e0439"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "658d824af28b0a8fedad6b0ca0743f59"
server_key_iv: "d692a6104d09c3015660c3050895bbf1"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.883080 8198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:44.890956 8198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:44.891191 8198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:44.891286 8198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:44.891448 8198 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:44.891682 8198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:44.891713 8198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:44.891733 8198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:44.891746 8198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:44.897444 8198 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.193:41725
I20260924 21:20:44.897511 8325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.193:41725 every 8 connection(s)
I20260924 21:20:44.897819 8198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/data/info.pb
I20260924 21:20:44.903004 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.898535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:58145 (local address 127.20.115.254:45001)
0924 21:20:44.898599 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:44.898601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:44.898822 (+ 221us) server_negotiation.cc:407] Connection header received
0924 21:20:44.899367 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:44.899368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:44.899385 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:44.899436 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:44.900121 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.900618 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.901676 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.901885 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.901973 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.902731 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:44.902732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:44.902732 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:44.902737 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:44.902752 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:44.902756 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:44.902809 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:44.902919 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 21:20:44.902942 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260924 21:20:44.903237 8328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:44.898537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45001 (local address 127.20.115.193:58145)
0924 21:20:44.898699 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:44.898732 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:44.899243 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:44.899507 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:44.899510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:44.899518 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:44.900010 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.900015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.900734 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.900735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.901579 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:44.901582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:44.901985 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:44.901986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:44.902112 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:44.902573 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:20:44.902574 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:44.902602 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:20:44.902640 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:44.902834 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:44.902841 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:44.902843 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:20:44.902907 (+ 64us) client_negotiation.cc:240] Negotiation successful
0924 21:20:44.902990 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260924 21:20:44.903915 8326 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45001
I20260924 21:20:44.904012 8326 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:44.904194 8326 heartbeater.cc:511] Master 127.20.115.254:45001 requested a full tablet report, sending...
I20260924 21:20:44.904667 8138 ts_manager.cc:194] Registered new tserver with Master: c80adeaf11d84b08a2f4fb74689e0439 (127.20.115.193:41725)
I20260924 21:20:44.905306 8138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.193:58145
I20260924 21:20:44.905390 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 8198
I20260924 21:20:44.905510 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-0/wal/instance
I20260924 21:20:44.905717 20943 external_mini_cluster.cc:1428] Setting key 4fa7a860d8a120a5c78741268a5e1573
I20260924 21:20:44.906556 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.194:0
--local_ip_for_outbound_sockets=127.20.115.194
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45001
--builtin_ntp_servers=127.20.115.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260924 21:20:44.985215 8329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:44.985423 8329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:44.985459 8329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:44.986610 8329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:44.986670 8329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:44.986711 8329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.194
I20260924 21:20:44.988031 8329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.115.194
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.194
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:44.988289 8329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:44.988560 8329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:44.993541 8334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:44.993541 8337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:44.993614 8329 server_base.cc:1034] running on GCE node
W20260924 21:20:44.993597 8335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:44.993948 8329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:44.994197 8329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:44.995362 8329 hybrid_clock.cc:648] HybridClock initialized: now 1790284844995344 us; error 35 us; skew 500 ppm
I20260924 21:20:44.996558 8329 webserver.cc:460] Webserver started at http://127.20.115.194:44607/ using document root <none> and password file <none>
I20260924 21:20:44.996869 8329 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:44.996948 8329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:44.997085 8329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:44.998104 8329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/data/instance:
uuid: "196341b0dac7421faa959b5740cf86ec"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "76db35860b95a17a9ebb41334812a778"
server_key_iv: "b9a512fdc44314c9542c28792dfd4b31"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.998469 8329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance:
uuid: "196341b0dac7421faa959b5740cf86ec"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "76db35860b95a17a9ebb41334812a778"
server_key_iv: "b9a512fdc44314c9542c28792dfd4b31"
server_key_version: "encryptionkey@0"
I20260924 21:20:44.999737 8329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:20:45.000466 8343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:45.000663 8329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:45.000753 8329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/wal
uuid: "196341b0dac7421faa959b5740cf86ec"
format_stamp: "Formatted at 2026-09-24 21:20:44 on dist-test-slave-cxpd"
server_key: "76db35860b95a17a9ebb41334812a778"
server_key_iv: "b9a512fdc44314c9542c28792dfd4b31"
server_key_version: "encryptionkey@0"
I20260924 21:20:45.000838 8329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:45.018548 8329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:45.018934 8329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:45.019098 8329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:45.019371 8329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:45.019687 8329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:45.019728 8329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:45.019752 8329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:45.019764 8329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:45.025394 8329 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.194:44723
I20260924 21:20:45.025468 8456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.194:44723 every 8 connection(s)
I20260924 21:20:45.025840 8329 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/data/info.pb
I20260924 21:20:45.031114 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.026566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.194:46089 (local address 127.20.115.254:45001)
0924 21:20:45.026611 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:45.026613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:45.026926 (+ 313us) server_negotiation.cc:407] Connection header received
0924 21:20:45.027470 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:45.027472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:45.027485 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:45.027537 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:45.028209 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.028711 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.029717 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.029908 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.029995 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.030818 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:45.030819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:45.030820 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:45.030825 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:45.030847 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:45.030851 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:45.030912 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:45.031005 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 21:20:45.031031 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:20:45.031286 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 8329
I20260924 21:20:45.031360 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.026581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45001 (local address 127.20.115.194:46089)
0924 21:20:45.026802 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:45.026835 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:45.027352 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:45.027612 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:45.027616 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:45.027623 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:45.028107 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.028111 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.028818 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.028819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.029609 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.029612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.030004 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.030005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.030136 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.030675 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:20:45.030676 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:45.030703 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:20:45.030727 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:45.030913 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:45.030920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:45.030923 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:45.031009 (+ 86us) client_negotiation.cc:240] Negotiation successful
0924 21:20:45.031109 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
I20260924 21:20:45.031410 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/wal/instance
I20260924 21:20:45.031622 20943 external_mini_cluster.cc:1428] Setting key 5cf11fac21bf8b50b4916b1962388d52
I20260924 21:20:45.032322 8457 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45001
I20260924 21:20:45.032424 8457 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:45.032629 8457 heartbeater.cc:511] Master 127.20.115.254:45001 requested a full tablet report, sending...
I20260924 21:20:45.032711 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.115.195:0
--local_ip_for_outbound_sockets=127.20.115.195
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45001
--builtin_ntp_servers=127.20.115.212:43461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260924 21:20:45.034056 8138 ts_manager.cc:194] Registered new tserver with Master: 196341b0dac7421faa959b5740cf86ec (127.20.115.194:44723)
I20260924 21:20:45.034437 8138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.194:46089
W20260924 21:20:45.113682 8460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:45.114037 8460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:45.114143 8460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:45.115361 8460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:45.115429 8460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:45.115475 8460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.115.195
I20260924 21:20:45.116950 8460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:43461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.115.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.115.195
--webserver_port=0
--tserver_master_addrs=127.20.115.254:45001
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.115.195
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:45.117214 8460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:45.117528 8460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:45.122522 8465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:45.122524 8466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:45.122560 8460 server_base.cc:1034] running on GCE node
W20260924 21:20:45.122525 8468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:45.122941 8460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:45.123190 8460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:45.124363 8460 hybrid_clock.cc:648] HybridClock initialized: now 1790284845124351 us; error 31 us; skew 500 ppm
I20260924 21:20:45.125594 8460 webserver.cc:460] Webserver started at http://127.20.115.195:39829/ using document root <none> and password file <none>
I20260924 21:20:45.125844 8460 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:45.125913 8460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:45.126065 8460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:45.127432 8460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/data/instance:
uuid: "53bfced21f1745ef865b5c98241ef213"
format_stamp: "Formatted at 2026-09-24 21:20:45 on dist-test-slave-cxpd"
server_key: "9518c795f138d34e8be64f59492aef34"
server_key_iv: "3b3413496b72e3fa1d014406417e917b"
server_key_version: "encryptionkey@0"
I20260924 21:20:45.127799 8460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance:
uuid: "53bfced21f1745ef865b5c98241ef213"
format_stamp: "Formatted at 2026-09-24 21:20:45 on dist-test-slave-cxpd"
server_key: "9518c795f138d34e8be64f59492aef34"
server_key_iv: "3b3413496b72e3fa1d014406417e917b"
server_key_version: "encryptionkey@0"
I20260924 21:20:45.129216 8460 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:20:45.129947 8474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:45.130110 8460 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:45.130227 8460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/wal
uuid: "53bfced21f1745ef865b5c98241ef213"
format_stamp: "Formatted at 2026-09-24 21:20:45 on dist-test-slave-cxpd"
server_key: "9518c795f138d34e8be64f59492aef34"
server_key_iv: "3b3413496b72e3fa1d014406417e917b"
server_key_version: "encryptionkey@0"
I20260924 21:20:45.130304 8460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:45.140086 8460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:45.140358 8460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:45.140496 8460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:45.140705 8460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260924 21:20:45.141032 8460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260924 21:20:45.141084 8460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:45.141134 8460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260924 21:20:45.141165 8460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:45.146570 8460 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.195:44881
I20260924 21:20:45.146641 8587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.195:44881 every 8 connection(s)
I20260924 21:20:45.146936 8460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/data/info.pb
I20260924 21:20:45.147390 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 8460
I20260924 21:20:45.147504 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/wal/instance
I20260924 21:20:45.147689 20943 external_mini_cluster.cc:1428] Setting key bf32edbfdb12f964a1cc65736300c51e
I20260924 21:20:45.152271 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.147933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.195:38427 (local address 127.20.115.254:45001)
0924 21:20:45.148036 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:45.148038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:45.148047 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:20:45.148589 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:45.148590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:45.148602 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:45.148650 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:45.149369 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.149893 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.150857 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.151095 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.151176 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.152017 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:45.152019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:45.152019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:45.152024 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:45.152039 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:45.152044 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:45.152098 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:45.152170 (+ 72us) server_negotiation.cc:299] Negotiation successful
0924 21:20:45.152194 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260924 21:20:45.152485 8590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.147646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:45001 (local address 127.20.115.195:38427)
0924 21:20:45.147823 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:45.147864 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:45.148498 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:45.148715 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:45.148719 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:45.148728 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:20:45.149274 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.149278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.149991 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.149992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.150765 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.150767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.151180 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.151181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.151329 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.151878 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0924 21:20:45.151879 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:20:45.151906 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0924 21:20:45.151927 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:20:45.152100 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:20:45.152105 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:20:45.152108 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:45.152173 (+ 65us) client_negotiation.cc:240] Negotiation successful
0924 21:20:45.152278 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260924 21:20:45.152916 8588 heartbeater.cc:347] Connected to a master server at 127.20.115.254:45001
I20260924 21:20:45.152995 8588 heartbeater.cc:464] Registering TS with master...
I20260924 21:20:45.153182 8588 heartbeater.cc:511] Master 127.20.115.254:45001 requested a full tablet report, sending...
I20260924 21:20:45.153551 8138 ts_manager.cc:194] Registered new tserver with Master: 53bfced21f1745ef865b5c98241ef213 (127.20.115.195:44881)
I20260924 21:20:45.153923 8138 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.115.195:38427
I20260924 21:20:45.159543 20943 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260924 21:20:45.163692 8177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.161144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46166 (local address 127.20.115.254:45001)
0924 21:20:45.161216 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:45.161218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:45.161336 (+ 118us) server_negotiation.cc:407] Connection header received
0924 21:20:45.161380 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:45.161381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:45.161390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:45.161427 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:45.161541 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.161982 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.162716 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.162890 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.162942 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.163485 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:45.163486 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:45.163486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:45.163489 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:45.163500 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:45.163504 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:45.163540 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:45.163579 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 21:20:45.163595 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260924 21:20:45.165051 8138 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46166:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260924 21:20:45.166550 8138 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260924 21:20:45.172356 8597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.168522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:44881 (local address 127.0.0.1:53954)
0924 21:20:45.168757 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:45.168767 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:45.168838 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:45.169091 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:45.169094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:45.169097 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:45.169155 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.169157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.170545 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.170546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.171275 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.171277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.172077 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.172078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.172153 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.172166 (+ 13us) client_negotiation.cc:769] Sending connection context
0924 21:20:45.172238 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:20:45.172285 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":89,"threads_started":1}
I20260924 21:20:45.172385 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.168744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53954 (local address 127.20.115.195:44881)
0924 21:20:45.168908 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:45.168912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:45.168921 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:20:45.168955 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:45.168962 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:45.168974 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:45.169066 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:45.169246 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.170372 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.171365 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.171997 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.172070 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.172146 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:45.172269 (+ 123us) server_negotiation.cc:299] Negotiation successful
0924 21:20:45.172304 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":103,"threads_started":1}
I20260924 21:20:45.173002 8596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.168484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:41725 (local address 127.0.0.1:55422)
0924 21:20:45.168595 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:45.168632 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:45.168719 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:45.169247 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:45.169248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:45.169251 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:45.169300 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.169301 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.170524 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.170525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.171655 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.171657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.172736 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.172737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.172816 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.172820 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:45.172881 (+ 61us) client_negotiation.cc:240] Negotiation successful
0924 21:20:45.172918 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":12,"thread_start_us":68,"threads_started":1}
I20260924 21:20:45.173094 8598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.168533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:44723 (local address 127.0.0.1:40540)
0924 21:20:45.168920 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:45.168929 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:45.168965 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:45.169454 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:45.169455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:45.169456 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:45.169492 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.169493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.170348 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.170349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.171655 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.171657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.172896 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.172897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.172945 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.172948 (+ 3us) client_negotiation.cc:769] Sending connection context
0924 21:20:45.172988 (+ 40us) client_negotiation.cc:240] Negotiation successful
0924 21:20:45.173022 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":31,"thread_start_us":73,"threads_started":1}
I20260924 21:20:45.173133 8522 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe22780881547e8b2f352aec1c63148 (DEFAULT_TABLE table=test-table [id=2a1c1439c97049d6b7dfb51870261b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:45.173306 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.168674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55422 (local address 127.20.115.193:41725)
0924 21:20:45.169074 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:45.169078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:45.169087 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:20:45.169130 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:45.169131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:45.169142 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:45.169373 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:45.169428 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.170455 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.171757 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.172629 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.172731 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.172829 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:45.173182 (+ 353us) server_negotiation.cc:299] Negotiation successful
0924 21:20:45.173235 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":76,"threads_started":1}
I20260924 21:20:45.173534 8601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.168935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40540 (local address 127.20.115.194:44723)
0924 21:20:45.169307 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:45.169311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:45.169320 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:20:45.169358 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:45.169359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:45.169370 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:45.169456 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:45.169547 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.170226 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.171753 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.172778 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.172897 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.172978 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:45.173004 (+ 26us) server_negotiation.cc:299] Negotiation successful
0924 21:20:45.173305 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":82,"threads_started":1}
I20260924 21:20:45.173525 8522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe22780881547e8b2f352aec1c63148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:45.173926 8260 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe22780881547e8b2f352aec1c63148 (DEFAULT_TABLE table=test-table [id=2a1c1439c97049d6b7dfb51870261b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:45.174248 8260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe22780881547e8b2f352aec1c63148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:45.174221 8391 tablet_service.cc:1467] Processing CreateTablet for tablet 1fe22780881547e8b2f352aec1c63148 (DEFAULT_TABLE table=test-table [id=2a1c1439c97049d6b7dfb51870261b3a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260924 21:20:45.174525 8391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1fe22780881547e8b2f352aec1c63148. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:45.176407 8602 tablet_bootstrap.cc:492] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: Bootstrap starting.
I20260924 21:20:45.177009 8602 tablet_bootstrap.cc:654] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:45.177309 8602 log.cc:824] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:45.177927 8605 tablet_bootstrap.cc:492] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: Bootstrap starting.
I20260924 21:20:45.178025 8603 tablet_bootstrap.cc:492] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439: Bootstrap starting.
I20260924 21:20:45.178160 8602 tablet_bootstrap.cc:492] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: No bootstrap required, opened a new log
I20260924 21:20:45.178256 8602 ts_tablet_manager.cc:1397] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:20:45.178582 8605 tablet_bootstrap.cc:654] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:45.178593 8603 tablet_bootstrap.cc:654] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:45.178913 8603 log.cc:824] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:45.178913 8605 log.cc:824] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:45.179684 8603 tablet_bootstrap.cc:492] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439: No bootstrap required, opened a new log
I20260924 21:20:45.179756 8603 ts_tablet_manager.cc:1397] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:20:45.179903 8605 tablet_bootstrap.cc:492] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: No bootstrap required, opened a new log
I20260924 21:20:45.179963 8605 ts_tablet_manager.cc:1397] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260924 21:20:45.180608 8602 raft_consensus.cc:348] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } }
I20260924 21:20:45.180773 8602 raft_consensus.cc:374] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:45.180836 8602 raft_consensus.cc:729] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53bfced21f1745ef865b5c98241ef213, State: Initialized, Role: FOLLOWER
I20260924 21:20:45.180949 8602 consensus_queue.cc:260] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } }
I20260924 21:20:45.181232 8602 ts_tablet_manager.cc:1428] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:20:45.181252 8588 heartbeater.cc:503] Master 127.20.115.254:45001 was elected leader, sending a full tablet report...
I20260924 21:20:45.182400 8603 raft_consensus.cc:348] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } }
I20260924 21:20:45.182562 8603 raft_consensus.cc:374] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:45.182596 8603 raft_consensus.cc:729] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c80adeaf11d84b08a2f4fb74689e0439, State: Initialized, Role: FOLLOWER
I20260924 21:20:45.182507 8605 raft_consensus.cc:348] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } }
I20260924 21:20:45.182637 8605 raft_consensus.cc:374] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:45.182677 8605 raft_consensus.cc:729] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 196341b0dac7421faa959b5740cf86ec, State: Initialized, Role: FOLLOWER
I20260924 21:20:45.182734 8603 consensus_queue.cc:260] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } }
I20260924 21:20:45.182789 8605 consensus_queue.cc:260] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } }
I20260924 21:20:45.182983 8603 ts_tablet_manager.cc:1428] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:20:45.183007 8605 ts_tablet_manager.cc:1428] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260924 21:20:45.183073 8457 heartbeater.cc:503] Master 127.20.115.254:45001 was elected leader, sending a full tablet report...
I20260924 21:20:45.183118 8326 heartbeater.cc:503] Master 127.20.115.254:45001 was elected leader, sending a full tablet report...
W20260924 21:20:45.276475 8458 tablet.cc:2369] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:20:45.369432 8609 raft_consensus.cc:482] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260924 21:20:45.369604 8609 raft_consensus.cc:504] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } }
I20260924 21:20:45.370110 8609 leader_election.cc:288] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53bfced21f1745ef865b5c98241ef213 (127.20.115.195:44881), 196341b0dac7421faa959b5740cf86ec (127.20.115.194:44723)
I20260924 21:20:45.374119 8600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.370382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:38971 (local address 127.20.115.195:44881)
0924 21:20:45.370428 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:45.370431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:45.370460 (+ 29us) server_negotiation.cc:407] Connection header received
0924 21:20:45.370492 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:45.370493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:45.370516 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:45.370580 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:45.370818 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.371671 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.373195 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.373757 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.373839 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.373847 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:45.374008 (+ 161us) server_negotiation.cc:299] Negotiation successful
0924 21:20:45.374049 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:20:45.374119 8601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.370402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.115.193:44259 (local address 127.20.115.194:44723)
0924 21:20:45.370448 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:45.370450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:45.370597 (+ 147us) server_negotiation.cc:407] Connection header received
0924 21:20:45.370707 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:45.370709 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:45.370720 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:45.370792 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0924 21:20:45.370963 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.371681 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.373195 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.373757 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.373839 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.373847 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:45.374004 (+ 157us) server_negotiation.cc:299] Negotiation successful
0924 21:20:45.374054 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:20:45.374125 8328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.370289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.195:44881 (local address 127.20.115.193:38971)
0924 21:20:45.370356 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:45.370369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:45.370422 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:45.370605 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:45.370607 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:45.370609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:45.370733 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.370735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.371752 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.371753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.373095 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.373098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.373847 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.373848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.373922 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.373926 (+ 4us) client_negotiation.cc:769] Sending connection context
0924 21:20:45.373984 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 21:20:45.374034 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":45}
I20260924 21:20:45.374131 8611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.370344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.194:44723 (local address 127.20.115.193:44259)
0924 21:20:45.370536 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:45.370546 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:20:45.370627 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:20:45.370819 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:20:45.370820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:20:45.370822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0924 21:20:45.370892 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.370894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.371770 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.371771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.373096 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:20:45.373098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.373849 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.373849 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:20:45.373928 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.373930 (+ 2us) client_negotiation.cc:769] Sending connection context
0924 21:20:45.373984 (+ 54us) client_negotiation.cc:240] Negotiation successful
0924 21:20:45.374029 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":63,"thread_start_us":92,"threads_started":1}
I20260924 21:20:45.374469 8542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe22780881547e8b2f352aec1c63148" candidate_uuid: "c80adeaf11d84b08a2f4fb74689e0439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53bfced21f1745ef865b5c98241ef213" is_pre_election: true
I20260924 21:20:45.374639 8542 raft_consensus.cc:2454] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c80adeaf11d84b08a2f4fb74689e0439 in term 0.
I20260924 21:20:45.374697 8411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe22780881547e8b2f352aec1c63148" candidate_uuid: "c80adeaf11d84b08a2f4fb74689e0439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196341b0dac7421faa959b5740cf86ec" is_pre_election: true
I20260924 21:20:45.374837 8411 raft_consensus.cc:2454] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c80adeaf11d84b08a2f4fb74689e0439 in term 0.
I20260924 21:20:45.374881 8213 leader_election.cc:302] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53bfced21f1745ef865b5c98241ef213, c80adeaf11d84b08a2f4fb74689e0439; no voters:
I20260924 21:20:45.375077 8609 raft_consensus.cc:2781] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260924 21:20:45.375167 8609 raft_consensus.cc:482] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260924 21:20:45.375195 8609 raft_consensus.cc:3037] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:45.375893 8609 raft_consensus.cc:504] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } }
I20260924 21:20:45.376051 8609 leader_election.cc:288] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [CANDIDATE]: Term 1 election: Requested vote from peers 53bfced21f1745ef865b5c98241ef213 (127.20.115.195:44881), 196341b0dac7421faa959b5740cf86ec (127.20.115.194:44723)
I20260924 21:20:45.376202 8411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe22780881547e8b2f352aec1c63148" candidate_uuid: "c80adeaf11d84b08a2f4fb74689e0439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "196341b0dac7421faa959b5740cf86ec"
I20260924 21:20:45.376241 8542 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1fe22780881547e8b2f352aec1c63148" candidate_uuid: "c80adeaf11d84b08a2f4fb74689e0439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53bfced21f1745ef865b5c98241ef213"
I20260924 21:20:45.376284 8411 raft_consensus.cc:3037] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:45.376307 8542 raft_consensus.cc:3037] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:45.377226 8411 raft_consensus.cc:2454] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c80adeaf11d84b08a2f4fb74689e0439 in term 1.
I20260924 21:20:45.377226 8542 raft_consensus.cc:2454] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c80adeaf11d84b08a2f4fb74689e0439 in term 1.
I20260924 21:20:45.377426 8215 leader_election.cc:302] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 196341b0dac7421faa959b5740cf86ec, c80adeaf11d84b08a2f4fb74689e0439; no voters:
I20260924 21:20:45.377543 8609 raft_consensus.cc:2781] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:45.377677 8609 raft_consensus.cc:686] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [term 1 LEADER]: Becoming Leader. State: Replica: c80adeaf11d84b08a2f4fb74689e0439, State: Running, Role: LEADER
I20260924 21:20:45.377780 8609 consensus_queue.cc:237] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } }
I20260924 21:20:45.378613 8138 catalog_manager.cc:5495] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 reported cstate change: term changed from 0 to 1, leader changed from <none> to c80adeaf11d84b08a2f4fb74689e0439 (127.20.115.193). New cstate: current_term: 1 leader_uuid: "c80adeaf11d84b08a2f4fb74689e0439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c80adeaf11d84b08a2f4fb74689e0439" member_type: VOTER last_known_addr { host: "127.20.115.193" port: 41725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 } health_report { overall_health: UNKNOWN } } }
W20260924 21:20:45.397424 8589 tablet.cc:2369] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260924 21:20:45.398454 8327 tablet.cc:2369] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260924 21:20:45.400731 8599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.398299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55430 (local address 127.20.115.193:41725)
0924 21:20:45.398366 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:45.398368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:45.398378 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:20:45.398456 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:45.398457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:45.398481 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:45.398555 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:20:45.398704 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.399167 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.399932 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:45.400129 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:45.400201 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:45.400379 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:20:45.400478 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:20:45.400574 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:45.400594 (+ 20us) server_negotiation.cc:299] Negotiation successful
0924 21:20:45.400615 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260924 21:20:45.402493 8542 raft_consensus.cc:1261] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [term 1 FOLLOWER]: Refusing update from remote peer c80adeaf11d84b08a2f4fb74689e0439: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:45.402495 8411 raft_consensus.cc:1261] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [term 1 FOLLOWER]: Refusing update from remote peer c80adeaf11d84b08a2f4fb74689e0439: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:20:45.402841 8612 consensus_queue.cc:1035] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [LEADER]: Connected to new peer: Peer: permanent_uuid: "196341b0dac7421faa959b5740cf86ec" member_type: VOTER last_known_addr { host: "127.20.115.194" port: 44723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:45.403007 8609 consensus_queue.cc:1035] T 1fe22780881547e8b2f352aec1c63148 P c80adeaf11d84b08a2f4fb74689e0439 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53bfced21f1745ef865b5c98241ef213" member_type: VOTER last_known_addr { host: "127.20.115.195" port: 44881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:20:45.407331 8137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:45.407435 8137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46166:
table { table_name: "test-table" } modify_external_catalogs: true
I20260924 21:20:45.408188 8137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04cd43c1d7f74ccea1a9bf4d3d1c5fec: Sending DeleteTablet for 3 replicas of tablet 1fe22780881547e8b2f352aec1c63148
I20260924 21:20:45.408540 8260 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe22780881547e8b2f352aec1c63148 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:45 UTC) from {username='slave'} at 127.0.0.1:55422
I20260924 21:20:45.408601 8522 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe22780881547e8b2f352aec1c63148 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:45 UTC) from {username='slave'} at 127.0.0.1:53954
I20260924 21:20:45.408811 8391 tablet_service.cc:1514] Processing DeleteTablet for tablet 1fe22780881547e8b2f352aec1c63148 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-24 21:20:45 UTC) from {username='slave'} at 127.0.0.1:40540
I20260924 21:20:45.408985 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 8198
I20260924 21:20:45.409021 8626 tablet_replica.cc:330] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: stopping tablet replica
I20260924 21:20:45.409123 8626 raft_consensus.cc:2229] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:45.409210 8626 raft_consensus.cc:2258] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:45.410262 8626 ts_tablet_manager.cc:1905] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:45.410944 8625 tablet_replica.cc:330] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: stopping tablet replica
I20260924 21:20:45.411018 8625 raft_consensus.cc:2229] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260924 21:20:45.411089 8625 raft_consensus.cc:2258] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260924 21:20:45.411474 8625 ts_tablet_manager.cc:1905] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260924 21:20:45.412248 8626 ts_tablet_manager.cc:1918] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:45.412324 8626 log.cc:1196] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-1/wal/wals/1fe22780881547e8b2f352aec1c63148
I20260924 21:20:45.412557 8626 ts_tablet_manager.cc:1939] T 1fe22780881547e8b2f352aec1c63148 P 196341b0dac7421faa959b5740cf86ec: Deleting consensus metadata
I20260924 21:20:45.412936 8625 ts_tablet_manager.cc:1918] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260924 21:20:45.413017 8625 log.cc:1196] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: Deleting WAL directory at /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790284606872664-20943-0/minicluster-data/ts-2/wal/wals/1fe22780881547e8b2f352aec1c63148
I20260924 21:20:45.413170 8124 catalog_manager.cc:4841] TS 196341b0dac7421faa959b5740cf86ec (127.20.115.194:44723): tablet 1fe22780881547e8b2f352aec1c63148 (table test-table [id=2a1c1439c97049d6b7dfb51870261b3a]) successfully deleted
I20260924 21:20:45.413272 8625 ts_tablet_manager.cc:1939] T 1fe22780881547e8b2f352aec1c63148 P 53bfced21f1745ef865b5c98241ef213: Deleting consensus metadata
I20260924 21:20:45.413669 8122 catalog_manager.cc:4841] TS 53bfced21f1745ef865b5c98241ef213 (127.20.115.195:44881): tablet 1fe22780881547e8b2f352aec1c63148 (table test-table [id=2a1c1439c97049d6b7dfb51870261b3a]) successfully deleted
W20260924 21:20:45.415043 8125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.115.193:41725 (error 108)
I20260924 21:20:45.415215 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 8329
I20260924 21:20:45.415858 8598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:45.415355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.193:41725 (local address 127.0.0.1:55440)
0924 21:20:45.415775 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0924 21:20:45.415815 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.193:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401}
W20260924 21:20:45.416317 8125 catalog_manager.cc:4584] TS c80adeaf11d84b08a2f4fb74689e0439 (127.20.115.193:41725): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1fe22780881547e8b2f352aec1c63148: Network error: Client connection negotiation failed: client connection to 127.20.115.193:41725: connect: Connection refused (error 111)
I20260924 21:20:45.420343 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 8460
I20260924 21:20:45.424870 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 8107
2026-09-24T21:20:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (847 ms)
[----------] 6 tests from EncryptionPolicyTest (8424 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:45 dist-test-slave-cxpd krb5kdc[8630](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:45 dist-test-slave-cxpd krb5kdc[8630](info): set up 2 sockets
Sep 24 21:20:45 dist-test-slave-cxpd krb5kdc[8630](info): commencing operation
krb5kdc: starting...
W20260924 21:20:46.460675 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:46 dist-test-slave-cxpd krb5kdc[8630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:46.573730 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37497
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:46.654105 8643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:46.654302 8643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:46.654325 8643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:46.655412 8643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:46.655452 8643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:46.655465 8643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:46.655478 8643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:46.655490 8643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:46.656900 8643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37497
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:46.657085 8643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:46.657253 8643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:46.662070 8648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:46.662071 8649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:46.662140 8643 server_base.cc:1034] running on GCE node
W20260924 21:20:46.662082 8651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:46.662493 8643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:46.662832 8643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:46.663997 8643 hybrid_clock.cc:648] HybridClock initialized: now 1790284846663992 us; error 36 us; skew 500 ppm
Sep 24 21:20:46 dist-test-slave-cxpd krb5kdc[8630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284846, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:46.666592 8643 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:46.666970 8643 webserver.cc:460] Webserver started at http://127.0.0.1:37321/ using document root <none> and password file <none>
I20260924 21:20:46.667208 8643 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:46.667256 8643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:46.667390 8643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:46.668427 8643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "844c5933a30a4cdc822694c031c037fe"
format_stamp: "Formatted at 2026-09-24 21:20:46 on dist-test-slave-cxpd"
server_key: "314f4387ede371239649fbf26de1c905"
server_key_iv: "28842e109c4c34a321999acf2973ec75"
server_key_version: "encryptionkey@0"
I20260924 21:20:46.668833 8643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "844c5933a30a4cdc822694c031c037fe"
format_stamp: "Formatted at 2026-09-24 21:20:46 on dist-test-slave-cxpd"
server_key: "314f4387ede371239649fbf26de1c905"
server_key_iv: "28842e109c4c34a321999acf2973ec75"
server_key_version: "encryptionkey@0"
I20260924 21:20:46.670234 8643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:20:46.671100 8658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:46.671315 8643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:20:46.671407 8643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "844c5933a30a4cdc822694c031c037fe"
format_stamp: "Formatted at 2026-09-24 21:20:46 on dist-test-slave-cxpd"
server_key: "314f4387ede371239649fbf26de1c905"
server_key_iv: "28842e109c4c34a321999acf2973ec75"
server_key_version: "encryptionkey@0"
I20260924 21:20:46.671461 8643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:46.714038 8643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:46.716413 8643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:46.716526 8643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:46.720508 8643 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37497
I20260924 21:20:46.720564 8710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37497 every 8 connection(s)
I20260924 21:20:46.720974 8643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:46.721873 8711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:46.724092 8711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe: Bootstrap starting.
I20260924 21:20:46.724680 8711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:46.724989 8711 log.cc:824] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:46.725860 8711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe: No bootstrap required, opened a new log
I20260924 21:20:46.727399 8711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844c5933a30a4cdc822694c031c037fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } }
I20260924 21:20:46.727562 8711 raft_consensus.cc:374] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:46.727622 8711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 844c5933a30a4cdc822694c031c037fe, State: Initialized, Role: FOLLOWER
I20260924 21:20:46.727790 8711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844c5933a30a4cdc822694c031c037fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } }
I20260924 21:20:46.727877 8711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:46.727921 8711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:46.727968 8711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:46.728492 8711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844c5933a30a4cdc822694c031c037fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } }
I20260924 21:20:46.728615 8711 leader_election.cc:302] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 844c5933a30a4cdc822694c031c037fe; no voters:
I20260924 21:20:46.728771 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 8643
I20260924 21:20:46.728839 8711 leader_election.cc:288] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:46.728866 8714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:46.728935 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:46.728986 8714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [term 1 LEADER]: Becoming Leader. State: Replica: 844c5933a30a4cdc822694c031c037fe, State: Running, Role: LEADER
I20260924 21:20:46.729112 8711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:46.729174 20943 external_mini_cluster.cc:1428] Setting key 1b6569adc7c95b09bc63d1d847cbe32f
I20260924 21:20:46.729130 8714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844c5933a30a4cdc822694c031c037fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } }
I20260924 21:20:46.729573 8716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 844c5933a30a4cdc822694c031c037fe. Latest consensus state: current_term: 1 leader_uuid: "844c5933a30a4cdc822694c031c037fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844c5933a30a4cdc822694c031c037fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } } }
I20260924 21:20:46.729655 8715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "844c5933a30a4cdc822694c031c037fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "844c5933a30a4cdc822694c031c037fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37497 } } }
I20260924 21:20:46.729728 8716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:46.729744 8715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:46.730635 8725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:46.731308 8725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:46.733635 8725 catalog_manager.cc:1347] Generated new cluster ID: 4bab752eb0d342efa4d0d298020f7c6d
I20260924 21:20:46.733691 8725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:20:46 dist-test-slave-cxpd krb5kdc[8630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284846, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 21:20:46.740098 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:46.729803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43927 (local address 127.0.0.1:37497)
0924 21:20:46.729993 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:46.730003 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:46.730078 (+ 75us) server_negotiation.cc:407] Connection header received
0924 21:20:46.730283 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:46.730291 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:46.730360 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:46.730500 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:46.731265 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:46.732453 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:46.733564 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:46.734020 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:46.734140 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:46.736433 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:46.736442 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:46.736442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:46.736452 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:46.738547 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:46.738948 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:46.738951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:46.738952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:46.739051 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:46.739284 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:46.739285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:46.739286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:46.739524 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:46.739715 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:46.739800 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 21:20:46.739899 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":63,"threads_started":1}
I20260924 21:20:46.741043 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:46.751029 8724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:46.742401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33727 (local address 127.0.0.1:37497)
0924 21:20:46.742439 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:46.742440 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:46.745005 (+ 2565us) server_negotiation.cc:407] Connection header received
0924 21:20:46.745072 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:46.745073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:46.745085 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:46.745164 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:46.745684 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:46.746200 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:46.746938 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:46.747167 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:46.747247 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:46.748420 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:46.748421 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:46.748422 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:46.748428 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:46.749848 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:46.750220 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:46.750222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:46.750222 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:46.750281 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:46.750505 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:46.750506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:46.750506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:46.750617 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:46.750743 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:46.750855 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 21:20:46.750913 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:20:46.753011 8725 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:46.753527 8725 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:46.758827 8725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 844c5933a30a4cdc822694c031c037fe: Generated new TSK 0
I20260924 21:20:46.759024 8725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:46.765215 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 8643
W20260924 21:20:46.769615 8628 connection.cc:535] client connection to 127.0.0.1:37497 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37497): Connection reset by peer (error 104)
2026-09-24T21:20:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1339 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:46 dist-test-slave-cxpd krb5kdc[8742](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:46 dist-test-slave-cxpd krb5kdc[8742](info): set up 2 sockets
Sep 24 21:20:46 dist-test-slave-cxpd krb5kdc[8742](info): commencing operation
krb5kdc: starting...
W20260924 21:20:47.808883 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:47 dist-test-slave-cxpd krb5kdc[8742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:47.929837 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42597
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:48.011880 8755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:48.012099 8755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:48.012123 8755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:48.013243 8755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:48.013280 8755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:48.013294 8755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:48.013306 8755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:48.013319 8755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:48.014756 8755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42597
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37975
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:48.014962 8755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:48.015154 8755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:48.020033 8760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:48.020040 8761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:48.020051 8763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:48.020484 8755 server_base.cc:1034] running on GCE node
I20260924 21:20:48.020725 8755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:48.021049 8755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:48.022213 8755 hybrid_clock.cc:648] HybridClock initialized: now 1790284848022201 us; error 33 us; skew 500 ppm
Sep 24 21:20:48 dist-test-slave-cxpd krb5kdc[8742](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284848, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:48.030262 8755 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:48.030689 8755 webserver.cc:460] Webserver started at http://127.0.0.1:37353/ using document root <none> and password file <none>
I20260924 21:20:48.030925 8755 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:48.030966 8755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:48.031059 8755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:48.032083 8755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "d0ea31ccef8d4523a73ecd6eae90fa01"
format_stamp: "Formatted at 2026-09-24 21:20:48 on dist-test-slave-cxpd"
server_key: "7740d7236f8017314bd56aa3e80c83a7"
server_key_iv: "3921f404436280e81593ce90b5229a00"
server_key_version: "encryptionkey@0"
I20260924 21:20:48.032377 8755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "d0ea31ccef8d4523a73ecd6eae90fa01"
format_stamp: "Formatted at 2026-09-24 21:20:48 on dist-test-slave-cxpd"
server_key: "7740d7236f8017314bd56aa3e80c83a7"
server_key_iv: "3921f404436280e81593ce90b5229a00"
server_key_version: "encryptionkey@0"
I20260924 21:20:48.033766 8755 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:20:48.034518 8770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:48.034675 8755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:20:48.034763 8755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "d0ea31ccef8d4523a73ecd6eae90fa01"
format_stamp: "Formatted at 2026-09-24 21:20:48 on dist-test-slave-cxpd"
server_key: "7740d7236f8017314bd56aa3e80c83a7"
server_key_iv: "3921f404436280e81593ce90b5229a00"
server_key_version: "encryptionkey@0"
I20260924 21:20:48.034822 8755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:48.046240 8755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:48.048332 8755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:48.048445 8755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:48.052485 8755 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37975
I20260924 21:20:48.052536 8822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37975 every 8 connection(s)
I20260924 21:20:48.052939 8755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:48.053846 8823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:48.054471 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 8755
I20260924 21:20:48.054602 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:48.054826 20943 external_mini_cluster.cc:1428] Setting key 5d6afd0945aa3d1b61ff4089c226a98d
I20260924 21:20:48.057349 8823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01: Bootstrap starting.
I20260924 21:20:48.058076 8823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:48.058372 8823 log.cc:824] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:48.059108 8823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01: No bootstrap required, opened a new log
I20260924 21:20:48.061335 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:48.055421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34367 (local address 127.0.0.1:37975)
0924 21:20:48.055662 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:48.055672 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:48.055707 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:20:48.056420 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:48.056430 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:48.056499 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:48.056666 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:48.056915 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:48.058346 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:48.059160 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:48.059748 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:48.059866 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:48.060557 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:48.060562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:48.060562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:48.060770 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:48.060834 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:48.060850 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:48.060943 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:48.061055 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 21:20:48.061083 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":127,"threads_started":1}
I20260924 21:20:48.061751 8823 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ea31ccef8d4523a73ecd6eae90fa01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } }
I20260924 21:20:48.061884 8823 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:48.061908 8823 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0ea31ccef8d4523a73ecd6eae90fa01, State: Initialized, Role: FOLLOWER
I20260924 21:20:48.062075 8823 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ea31ccef8d4523a73ecd6eae90fa01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } }
I20260924 21:20:48.062161 8823 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:48.062193 8823 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:48.062237 8823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:48.062502 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:48.062951 8823 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ea31ccef8d4523a73ecd6eae90fa01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } }
I20260924 21:20:48.063076 8823 leader_election.cc:302] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0ea31ccef8d4523a73ecd6eae90fa01; no voters:
I20260924 21:20:48.063241 8823 leader_election.cc:288] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:48.063300 8828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:48.063508 8828 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [term 1 LEADER]: Becoming Leader. State: Replica: d0ea31ccef8d4523a73ecd6eae90fa01, State: Running, Role: LEADER
I20260924 21:20:48.063510 8823 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:48.063673 8828 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ea31ccef8d4523a73ecd6eae90fa01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } }
I20260924 21:20:48.064076 8829 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0ea31ccef8d4523a73ecd6eae90fa01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ea31ccef8d4523a73ecd6eae90fa01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } } }
I20260924 21:20:48.064160 8829 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:48.064308 8829 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0ea31ccef8d4523a73ecd6eae90fa01. Latest consensus state: current_term: 1 leader_uuid: "d0ea31ccef8d4523a73ecd6eae90fa01" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0ea31ccef8d4523a73ecd6eae90fa01" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37975 } } }
I20260924 21:20:48.064363 8829 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:48.064533 8841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:48.065070 8841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:48.068485 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:48.065233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60053 (local address 127.0.0.1:37975)
0924 21:20:48.065273 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:48.065274 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:48.065565 (+ 291us) server_negotiation.cc:407] Connection header received
0924 21:20:48.065639 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:48.065640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:48.065649 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:48.065694 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:48.065843 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:48.066325 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:48.067241 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:48.067457 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:48.067535 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:48.068256 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:48.068257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:48.068257 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:48.068262 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:48.068277 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:48.068281 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:48.068336 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:48.068397 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 21:20:48.068406 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:20:48.070067 8841 catalog_manager.cc:1347] Generated new cluster ID: 4ec5ad5d7f7e4cd4a29fb8b24f42d78b
I20260924 21:20:48.070143 8841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:48.084462 8841 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:48.085062 8841 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:48.091809 8841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0ea31ccef8d4523a73ecd6eae90fa01: Generated new TSK 0
I20260924 21:20:48.092046 8841 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:48.098806 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 8755
W20260924 21:20:48.103372 8740 connection.cc:535] client connection to 127.0.0.1:37975 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37975): Connection reset by peer (error 104)
2026-09-24T21:20:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:48 dist-test-slave-cxpd krb5kdc[8854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:48 dist-test-slave-cxpd krb5kdc[8854](info): set up 2 sockets
Sep 24 21:20:48 dist-test-slave-cxpd krb5kdc[8854](info): commencing operation
krb5kdc: starting...
W20260924 21:20:49.140682 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:49 dist-test-slave-cxpd krb5kdc[8854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284849, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:49.255784 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33341
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:49.336982 8867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:49.337211 8867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:49.337248 8867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:49.338376 8867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:49.338443 8867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:49.338469 8867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:49.338502 8867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:49.338527 8867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:49.339982 8867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33341
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:49.340255 8867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:49.340473 8867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:49.345263 8872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:49.345340 8867 server_base.cc:1034] running on GCE node
W20260924 21:20:49.345294 8875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:49.345284 8873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:49.345732 8867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:49.346024 8867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:49.347201 8867 hybrid_clock.cc:648] HybridClock initialized: now 1790284849347195 us; error 50 us; skew 500 ppm
Sep 24 21:20:49 dist-test-slave-cxpd krb5kdc[8854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284849, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:49.349726 8867 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:49.350178 8867 webserver.cc:460] Webserver started at http://127.0.0.1:34775/ using document root <none> and password file <none>
I20260924 21:20:49.350353 8867 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:49.350391 8867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:49.350476 8867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:49.351438 8867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "a3f4cb1b4c8a4320992174e4eac23a16"
format_stamp: "Formatted at 2026-09-24 21:20:49 on dist-test-slave-cxpd"
server_key: "f48b26026631798c14936dc5d1aa2f55"
server_key_iv: "6f6bda58d42f3932510aa0a88eceacbd"
server_key_version: "encryptionkey@0"
I20260924 21:20:49.351728 8867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "a3f4cb1b4c8a4320992174e4eac23a16"
format_stamp: "Formatted at 2026-09-24 21:20:49 on dist-test-slave-cxpd"
server_key: "f48b26026631798c14936dc5d1aa2f55"
server_key_iv: "6f6bda58d42f3932510aa0a88eceacbd"
server_key_version: "encryptionkey@0"
I20260924 21:20:49.352972 8867 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:20:49.353811 8882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:49.354018 8867 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:49.354137 8867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "a3f4cb1b4c8a4320992174e4eac23a16"
format_stamp: "Formatted at 2026-09-24 21:20:49 on dist-test-slave-cxpd"
server_key: "f48b26026631798c14936dc5d1aa2f55"
server_key_iv: "6f6bda58d42f3932510aa0a88eceacbd"
server_key_version: "encryptionkey@0"
I20260924 21:20:49.354278 8867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:49.357576 8867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:49.359365 8867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:49.359511 8867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:49.363255 8867 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33341
I20260924 21:20:49.363282 8934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33341 every 8 connection(s)
I20260924 21:20:49.363598 8867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:49.364483 8935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:49.366761 8935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16: Bootstrap starting.
I20260924 21:20:49.367322 8935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:49.367612 8935 log.cc:824] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:49.368290 8935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16: No bootstrap required, opened a new log
I20260924 21:20:49.369920 8935 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f4cb1b4c8a4320992174e4eac23a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33341 } }
I20260924 21:20:49.370051 8935 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:49.370074 8935 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f4cb1b4c8a4320992174e4eac23a16, State: Initialized, Role: FOLLOWER
I20260924 21:20:49.370220 8935 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f4cb1b4c8a4320992174e4eac23a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33341 } }
I20260924 21:20:49.370303 8935 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:49.370347 8935 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:49.370386 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 8867
I20260924 21:20:49.370419 8935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:49.370505 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:49.370718 20943 external_mini_cluster.cc:1428] Setting key dea10c284c1b53a63eb947effb80057f
I20260924 21:20:49.371088 8935 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f4cb1b4c8a4320992174e4eac23a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33341 } }
I20260924 21:20:49.371263 8935 leader_election.cc:302] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3f4cb1b4c8a4320992174e4eac23a16; no voters:
I20260924 21:20:49.371524 8935 leader_election.cc:288] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:49.371567 8940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:49.371762 8940 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [term 1 LEADER]: Becoming Leader. State: Replica: a3f4cb1b4c8a4320992174e4eac23a16, State: Running, Role: LEADER
I20260924 21:20:49.371863 8935 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:49.371898 8940 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f4cb1b4c8a4320992174e4eac23a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33341 } }
I20260924 21:20:49.372293 8940 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3f4cb1b4c8a4320992174e4eac23a16. Latest consensus state: current_term: 1 leader_uuid: "a3f4cb1b4c8a4320992174e4eac23a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f4cb1b4c8a4320992174e4eac23a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33341 } } }
I20260924 21:20:49.372385 8940 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:49.372437 8942 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3f4cb1b4c8a4320992174e4eac23a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f4cb1b4c8a4320992174e4eac23a16" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33341 } } }
I20260924 21:20:49.372999 8942 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:49.372642 8944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:49.373096 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:49.371238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35027 (local address 127.0.0.1:33341)
0924 21:20:49.371490 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:49.371499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:49.371536 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:20:49.371666 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:49.371674 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:49.371708 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:49.371841 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:49.372526 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:49.372529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:49.372530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:49.372534 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:49.372586 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:49.372601 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:49.372664 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:49.372848 (+ 184us) server_negotiation.cc:299] Negotiation successful
0924 21:20:49.372887 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260924 21:20:49.373627 8944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:49.374131 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:49.375437 8944 catalog_manager.cc:1347] Generated new cluster ID: 9e24d53c60c9498d9cfa1454d523d4f2
I20260924 21:20:49.375478 8944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:49.381886 8939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:49.377155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60167 (local address 127.0.0.1:33341)
0924 21:20:49.377189 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:49.377191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:49.380955 (+ 3764us) server_negotiation.cc:407] Connection header received
0924 21:20:49.380989 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:49.380989 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:49.380995 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:49.381060 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:49.381661 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:49.381662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:49.381662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:49.381666 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:49.381681 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:49.381686 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:49.381753 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:49.381797 (+ 44us) server_negotiation.cc:299] Negotiation successful
0924 21:20:49.381810 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:20:49.398808 8944 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:49.398986 8944 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:49.402220 8944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3f4cb1b4c8a4320992174e4eac23a16: Generated new TSK 0
I20260924 21:20:49.402395 8944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:49.404549 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 8867
2026-09-24T21:20:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1305 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:49 dist-test-slave-cxpd krb5kdc[8966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:49 dist-test-slave-cxpd krb5kdc[8966](info): set up 2 sockets
Sep 24 21:20:49 dist-test-slave-cxpd krb5kdc[8966](info): commencing operation
krb5kdc: starting...
W20260924 21:20:50.440733 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:50 dist-test-slave-cxpd krb5kdc[8966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:50.553154 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44031
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:50.631037 8979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:50.631253 8979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:50.631290 8979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:50.632395 8979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:50.632464 8979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:50.632488 8979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:50.632515 8979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:50.632539 8979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:50.633975 8979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:50.634229 8979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:50.634449 8979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:50.639066 8987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:50.639039 8985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:50.639047 8984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:50.639387 8979 server_base.cc:1034] running on GCE node
I20260924 21:20:50.639652 8979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:50.639918 8979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:50.641081 8979 hybrid_clock.cc:648] HybridClock initialized: now 1790284850641063 us; error 29 us; skew 500 ppm
Sep 24 21:20:50 dist-test-slave-cxpd krb5kdc[8966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284850, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:50.643460 8979 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:50.643864 8979 webserver.cc:460] Webserver started at http://127.0.0.1:35779/ using document root <none> and password file <none>
I20260924 21:20:50.644088 8979 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:50.644164 8979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:50.644315 8979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:50.645368 8979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "8f46fba068fd4c7e92434eeae908ffcc"
format_stamp: "Formatted at 2026-09-24 21:20:50 on dist-test-slave-cxpd"
server_key: "98cb39a84b439d83cbd25c4a64f79f56"
server_key_iv: "e352d279952fdcc767d68bf04e11f19b"
server_key_version: "encryptionkey@0"
I20260924 21:20:50.645715 8979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "8f46fba068fd4c7e92434eeae908ffcc"
format_stamp: "Formatted at 2026-09-24 21:20:50 on dist-test-slave-cxpd"
server_key: "98cb39a84b439d83cbd25c4a64f79f56"
server_key_iv: "e352d279952fdcc767d68bf04e11f19b"
server_key_version: "encryptionkey@0"
I20260924 21:20:50.647030 8979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:20:50.647684 8994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:50.647852 8979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:50.647948 8979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "8f46fba068fd4c7e92434eeae908ffcc"
format_stamp: "Formatted at 2026-09-24 21:20:50 on dist-test-slave-cxpd"
server_key: "98cb39a84b439d83cbd25c4a64f79f56"
server_key_iv: "e352d279952fdcc767d68bf04e11f19b"
server_key_version: "encryptionkey@0"
I20260924 21:20:50.648025 8979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:50.657884 8979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:50.660017 8979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:50.660177 8979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:50.663901 8979 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44031
I20260924 21:20:50.663930 9046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44031 every 8 connection(s)
I20260924 21:20:50.664247 8979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:50.665112 9047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:50.666864 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 8979
I20260924 21:20:50.666999 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:50.667207 20943 external_mini_cluster.cc:1428] Setting key b2e113826169b7a9e1f876604eddb57c
I20260924 21:20:50.667335 9047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc: Bootstrap starting.
I20260924 21:20:50.668035 9047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:50.668330 9047 log.cc:824] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:50.669165 9047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc: No bootstrap required, opened a new log
I20260924 21:20:50.671362 9047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f46fba068fd4c7e92434eeae908ffcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44031 } }
I20260924 21:20:50.671533 9047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:50.671569 9047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f46fba068fd4c7e92434eeae908ffcc, State: Initialized, Role: FOLLOWER
I20260924 21:20:50.671675 9047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f46fba068fd4c7e92434eeae908ffcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44031 } }
I20260924 21:20:50.671743 9047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:50.671772 9047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:50.671814 9047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:50.672456 9047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f46fba068fd4c7e92434eeae908ffcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44031 } }
I20260924 21:20:50.672574 9047 leader_election.cc:302] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f46fba068fd4c7e92434eeae908ffcc; no voters:
I20260924 21:20:50.672744 9047 leader_election.cc:288] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:50.672765 9052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:50.672899 9052 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [term 1 LEADER]: Becoming Leader. State: Replica: 8f46fba068fd4c7e92434eeae908ffcc, State: Running, Role: LEADER
I20260924 21:20:50.672974 9052 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f46fba068fd4c7e92434eeae908ffcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44031 } }
I20260924 21:20:50.673005 9047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:50.673278 9052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f46fba068fd4c7e92434eeae908ffcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f46fba068fd4c7e92434eeae908ffcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44031 } } }
I20260924 21:20:50.673410 9052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:50.673585 9052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f46fba068fd4c7e92434eeae908ffcc. Latest consensus state: current_term: 1 leader_uuid: "8f46fba068fd4c7e92434eeae908ffcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f46fba068fd4c7e92434eeae908ffcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44031 } } }
I20260924 21:20:50.673717 9052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:50.674044 9059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 24 21:20:50 dist-test-slave-cxpd krb5kdc[8966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 21:20:50.674538 9059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:50.678932 9050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:50.667705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47269 (local address 127.0.0.1:44031)
0924 21:20:50.667867 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:50.667880 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:50.667903 (+ 23us) server_negotiation.cc:407] Connection header received
0924 21:20:50.668345 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:50.668352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:50.668397 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:50.668560 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:50.669343 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:50.670708 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:50.671469 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:50.672072 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:50.672179 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:50.675107 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:50.675111 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:50.675111 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:50.675122 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:50.677218 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:50.677648 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:50.677650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:50.677651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:50.677758 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:50.677980 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:50.677981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:50.677981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:50.678282 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:50.678416 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:50.678586 (+ 170us) server_negotiation.cc:299] Negotiation successful
0924 21:20:50.678722 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
I20260924 21:20:50.680397 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:50.681345 9059 catalog_manager.cc:1347] Generated new cluster ID: e0c63a148ed0461b963578cd134e435c
I20260924 21:20:50.681404 9059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:50.689435 9050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:50.681924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53583 (local address 127.0.0.1:44031)
0924 21:20:50.681958 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:50.681960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:50.682118 (+ 158us) server_negotiation.cc:407] Connection header received
0924 21:20:50.682207 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:50.682207 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:50.682217 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:50.682299 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:50.682941 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:50.683872 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:50.684611 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:50.684955 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:50.685020 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:50.686484 (+ 1464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:50.686485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:50.686486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:50.686494 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:50.688184 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:50.688572 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:50.688573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:50.688573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:50.688638 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:50.688876 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:50.688877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:50.688877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:50.689000 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:50.689072 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:50.689280 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 21:20:50.689351 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:20:50.698035 9059 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:50.698536 9059 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:50.703902 9059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f46fba068fd4c7e92434eeae908ffcc: Generated new TSK 0
I20260924 21:20:50.704077 9059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:50.708700 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 8979
2026-09-24T21:20:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1303 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:50 dist-test-slave-cxpd krb5kdc[9078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:50 dist-test-slave-cxpd krb5kdc[9078](info): set up 2 sockets
Sep 24 21:20:50 dist-test-slave-cxpd krb5kdc[9078](info): commencing operation
krb5kdc: starting...
W20260924 21:20:51.744899 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:51 dist-test-slave-cxpd krb5kdc[9078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284851, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:51.864559 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34895
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:51.944950 9091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:51.945160 9091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:51.945183 9091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:51.946324 9091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:51.946360 9091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:51.946374 9091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:51.946386 9091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:51.946398 9091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:51.947801 9091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:51.947975 9091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:51.948150 9091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:51.953022 9099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:51.953088 9096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:51.953017 9091 server_base.cc:1034] running on GCE node
W20260924 21:20:51.953017 9097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:51.953478 9091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:51.953807 9091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:51.954969 9091 hybrid_clock.cc:648] HybridClock initialized: now 1790284851954949 us; error 39 us; skew 500 ppm
Sep 24 21:20:51 dist-test-slave-cxpd krb5kdc[9078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284851, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:51.957530 9091 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:51.957959 9091 webserver.cc:460] Webserver started at http://127.0.0.1:43639/ using document root <none> and password file <none>
I20260924 21:20:51.958185 9091 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:51.958256 9091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:51.958407 9091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:51.959458 9091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "f77caf02e1f7435b9f0202c8357fe46b"
format_stamp: "Formatted at 2026-09-24 21:20:51 on dist-test-slave-cxpd"
server_key: "54bce14f17c7f6a518494cddeb4d54cf"
server_key_iv: "b779fc3aa6059a339d7659ef3d60cd93"
server_key_version: "encryptionkey@0"
I20260924 21:20:51.959823 9091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "f77caf02e1f7435b9f0202c8357fe46b"
format_stamp: "Formatted at 2026-09-24 21:20:51 on dist-test-slave-cxpd"
server_key: "54bce14f17c7f6a518494cddeb4d54cf"
server_key_iv: "b779fc3aa6059a339d7659ef3d60cd93"
server_key_version: "encryptionkey@0"
I20260924 21:20:51.961311 9091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:20:51.962072 9106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:51.962280 9091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:51.962386 9091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "f77caf02e1f7435b9f0202c8357fe46b"
format_stamp: "Formatted at 2026-09-24 21:20:51 on dist-test-slave-cxpd"
server_key: "54bce14f17c7f6a518494cddeb4d54cf"
server_key_iv: "b779fc3aa6059a339d7659ef3d60cd93"
server_key_version: "encryptionkey@0"
I20260924 21:20:51.962460 9091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:51.989214 9091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:51.991540 9091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:51.991707 9091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:51.995615 9091 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34895
I20260924 21:20:51.995657 9158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34895 every 8 connection(s)
I20260924 21:20:51.996032 9091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:51.996955 9159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:51.999135 9159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b: Bootstrap starting.
I20260924 21:20:51.999418 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 9091
I20260924 21:20:51.999550 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:51.999730 20943 external_mini_cluster.cc:1428] Setting key 7e96cb653deddc8f326366f7c1677ee5
I20260924 21:20:51.999891 9159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:52.000227 9159 log.cc:824] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:52.000988 9159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b: No bootstrap required, opened a new log
I20260924 21:20:52.003401 9159 raft_consensus.cc:348] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77caf02e1f7435b9f0202c8357fe46b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } }
I20260924 21:20:52.003536 9159 raft_consensus.cc:374] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:52.003561 9159 raft_consensus.cc:729] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77caf02e1f7435b9f0202c8357fe46b, State: Initialized, Role: FOLLOWER
I20260924 21:20:52.003712 9159 consensus_queue.cc:260] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77caf02e1f7435b9f0202c8357fe46b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } }
I20260924 21:20:52.003788 9159 raft_consensus.cc:388] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:52.003811 9159 raft_consensus.cc:482] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:52.003878 9159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:52.004561 9159 raft_consensus.cc:504] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77caf02e1f7435b9f0202c8357fe46b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } }
I20260924 21:20:52.004680 9159 leader_election.cc:302] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f77caf02e1f7435b9f0202c8357fe46b; no voters:
I20260924 21:20:52.004874 9159 leader_election.cc:288] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:52.004892 9164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:52.005002 9164 raft_consensus.cc:686] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [term 1 LEADER]: Becoming Leader. State: Replica: f77caf02e1f7435b9f0202c8357fe46b, State: Running, Role: LEADER
I20260924 21:20:52.005103 9164 consensus_queue.cc:237] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77caf02e1f7435b9f0202c8357fe46b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } }
I20260924 21:20:52.005141 9159 sys_catalog.cc:564] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:52.005411 9164 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f77caf02e1f7435b9f0202c8357fe46b. Latest consensus state: current_term: 1 leader_uuid: "f77caf02e1f7435b9f0202c8357fe46b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77caf02e1f7435b9f0202c8357fe46b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } } }
I20260924 21:20:52.005549 9164 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:52.005442 9166 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f77caf02e1f7435b9f0202c8357fe46b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77caf02e1f7435b9f0202c8357fe46b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34895 } } }
I20260924 21:20:52.005805 9166 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:52.005978 9171 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:52.006264 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:52.000263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40255 (local address 127.0.0.1:34895)
0924 21:20:52.000477 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:52.000485 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:52.000510 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:20:52.001143 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:52.001152 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:52.001214 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:52.001359 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:52.001576 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:52.003003 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:52.003915 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:52.004542 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:52.004655 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:52.005519 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:52.005523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:52.005523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:52.005746 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:52.005784 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:52.005800 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:52.005843 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:52.005984 (+ 141us) server_negotiation.cc:299] Negotiation successful
0924 21:20:52.006054 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":78,"threads_started":1}
I20260924 21:20:52.006573 9171 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:52.007685 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:52.008127 9171 catalog_manager.cc:1347] Generated new cluster ID: 19ef7d85aa93421885c80fb34232e6b2
I20260924 21:20:52.008194 9171 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:52.016678 9162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:52.013116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49083 (local address 127.0.0.1:34895)
0924 21:20:52.013156 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:52.013158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:52.013373 (+ 215us) server_negotiation.cc:407] Connection header received
0924 21:20:52.013424 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:52.013425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:52.013437 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:52.013497 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:52.013628 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:52.014368 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:52.015386 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:52.015671 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:52.015746 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:52.016428 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:52.016430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:52.016430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:52.016434 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:52.016450 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:52.016455 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:52.016513 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:52.016571 (+ 58us) server_negotiation.cc:299] Negotiation successful
0924 21:20:52.016594 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:20:52.023147 9171 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:52.023648 9171 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:52.026497 9171 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f77caf02e1f7435b9f0202c8357fe46b: Generated new TSK 0
I20260924 21:20:52.026660 9171 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:52.030484 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 9091
2026-09-24T21:20:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:52 dist-test-slave-cxpd krb5kdc[9190](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:52 dist-test-slave-cxpd krb5kdc[9190](info): set up 2 sockets
Sep 24 21:20:52 dist-test-slave-cxpd krb5kdc[9190](info): commencing operation
krb5kdc: starting...
W20260924 21:20:53.056983 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.009s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:53 dist-test-slave-cxpd krb5kdc[9190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:53.175467 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43643
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:53.255406 9203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:53.255667 9203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:53.255708 9203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:53.256903 9203 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:53.256981 9203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:53.257005 9203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:53.257035 9203 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:53.257066 9203 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:53.258599 9203 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:53.258888 9203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:53.259126 9203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:53.263994 9209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:53.264137 9211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:53.264204 9208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:53.264360 9203 server_base.cc:1034] running on GCE node
I20260924 21:20:53.264616 9203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:53.264961 9203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:53.266121 9203 hybrid_clock.cc:648] HybridClock initialized: now 1790284853266113 us; error 34 us; skew 500 ppm
Sep 24 21:20:53 dist-test-slave-cxpd krb5kdc[9190](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284853, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:53.277462 9203 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:53.277976 9203 webserver.cc:460] Webserver started at http://127.0.0.1:37955/ using document root <none> and password file <none>
I20260924 21:20:53.278240 9203 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:53.278288 9203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:53.278456 9203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:53.279554 9203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "ce0173f8098a4f9d8a661fc34d9c7537"
format_stamp: "Formatted at 2026-09-24 21:20:53 on dist-test-slave-cxpd"
server_key: "1eca75f782956eb2e511bf2ec3e838bc"
server_key_iv: "372ad16ccca56f285a80606118cc7986"
server_key_version: "encryptionkey@0"
I20260924 21:20:53.279923 9203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "ce0173f8098a4f9d8a661fc34d9c7537"
format_stamp: "Formatted at 2026-09-24 21:20:53 on dist-test-slave-cxpd"
server_key: "1eca75f782956eb2e511bf2ec3e838bc"
server_key_iv: "372ad16ccca56f285a80606118cc7986"
server_key_version: "encryptionkey@0"
I20260924 21:20:53.281327 9203 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:20:53.282119 9218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:53.282325 9203 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:53.282404 9203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "ce0173f8098a4f9d8a661fc34d9c7537"
format_stamp: "Formatted at 2026-09-24 21:20:53 on dist-test-slave-cxpd"
server_key: "1eca75f782956eb2e511bf2ec3e838bc"
server_key_iv: "372ad16ccca56f285a80606118cc7986"
server_key_version: "encryptionkey@0"
I20260924 21:20:53.282491 9203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:53.285849 9203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:53.287717 9203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:53.287869 9203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:53.291970 9203 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43643
I20260924 21:20:53.292004 9270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43643 every 8 connection(s)
I20260924 21:20:53.292366 9203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:53.293311 9271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:53.295555 9271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537: Bootstrap starting.
I20260924 21:20:53.296087 9271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:53.296312 9271 log.cc:824] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:53.297081 9271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537: No bootstrap required, opened a new log
I20260924 21:20:53.298666 9271 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0173f8098a4f9d8a661fc34d9c7537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43643 } }
I20260924 21:20:53.298804 9271 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:53.298827 9271 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce0173f8098a4f9d8a661fc34d9c7537, State: Initialized, Role: FOLLOWER
I20260924 21:20:53.298972 9271 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0173f8098a4f9d8a661fc34d9c7537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43643 } }
I20260924 21:20:53.299052 9271 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:53.299075 9271 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:53.299110 9271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:53.299592 9271 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0173f8098a4f9d8a661fc34d9c7537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43643 } }
I20260924 21:20:53.299678 9271 leader_election.cc:302] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce0173f8098a4f9d8a661fc34d9c7537; no voters:
I20260924 21:20:53.299798 9271 leader_election.cc:288] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:53.299981 9275 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0173f8098a4f9d8a661fc34d9c7537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43643 } } }
I20260924 21:20:53.300051 9271 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:53.300134 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 9203
I20260924 21:20:53.300057 9275 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:20:53.300256 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:53.300459 20943 external_mini_cluster.cc:1428] Setting key 34e05fdda8bf4498cf3b9504e9c21296
I20260924 21:20:53.300808 9274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:53.300927 9274 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [term 1 LEADER]: Becoming Leader. State: Replica: ce0173f8098a4f9d8a661fc34d9c7537, State: Running, Role: LEADER
I20260924 21:20:53.301062 9274 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0173f8098a4f9d8a661fc34d9c7537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43643 } }
I20260924 21:20:53.301446 9275 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce0173f8098a4f9d8a661fc34d9c7537. Latest consensus state: current_term: 1 leader_uuid: "ce0173f8098a4f9d8a661fc34d9c7537" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce0173f8098a4f9d8a661fc34d9c7537" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43643 } } }
I20260924 21:20:53.301580 9275 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:53.301805 9288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:53.302636 9288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:53.303418 9281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:53.301030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55917 (local address 127.0.0.1:43643)
0924 21:20:53.301846 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:53.301856 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:53.301895 (+ 39us) server_negotiation.cc:407] Connection header received
0924 21:20:53.302025 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:53.302032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:53.302061 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:53.302236 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:53.302982 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:53.302985 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:53.302986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:53.302990 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:53.303035 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:53.303046 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:53.303124 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:53.303189 (+ 65us) server_negotiation.cc:299] Negotiation successful
0924 21:20:53.303229 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":92,"threads_started":1}
I20260924 21:20:53.304371 9288 catalog_manager.cc:1347] Generated new cluster ID: 060254318881457195a0b5bdbf61a856
I20260924 21:20:53.304388 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:53.304425 9288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:53.307005 9281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:53.305855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47099 (local address 127.0.0.1:43643)
0924 21:20:53.305887 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:53.305889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:53.306037 (+ 148us) server_negotiation.cc:407] Connection header received
0924 21:20:53.306093 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:53.306094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:53.306100 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:53.306172 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:53.306770 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:53.306772 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:53.306772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:53.306776 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:53.306792 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:53.306797 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:53.306860 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:53.306913 (+ 53us) server_negotiation.cc:299] Negotiation successful
0924 21:20:53.306929 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:20:53.318567 9288 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:53.318740 9288 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:53.323756 9288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce0173f8098a4f9d8a661fc34d9c7537: Generated new TSK 0
I20260924 21:20:53.323962 9288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:53.329366 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 9203
2026-09-24T21:20:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:53 dist-test-slave-cxpd krb5kdc[9301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:53 dist-test-slave-cxpd krb5kdc[9301](info): set up 2 sockets
Sep 24 21:20:53 dist-test-slave-cxpd krb5kdc[9301](info): commencing operation
krb5kdc: starting...
W20260924 21:20:54.372859 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:54 dist-test-slave-cxpd krb5kdc[9301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:54.489552 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:54.568432 9314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:54.568638 9314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:54.568661 9314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:54.569842 9314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:54.569886 9314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:54.569900 9314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:54.569913 9314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:54.569926 9314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:54.571439 9314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:54.571702 9314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:54.571936 9314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:54.576725 9319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:54.576864 9314 server_base.cc:1034] running on GCE node
W20260924 21:20:54.576730 9320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:54.576938 9322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:54.577234 9314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:54.577554 9314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:54.578727 9314 hybrid_clock.cc:648] HybridClock initialized: now 1790284854578707 us; error 39 us; skew 500 ppm
Sep 24 21:20:54 dist-test-slave-cxpd krb5kdc[9301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284854, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:54.581264 9314 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:54.581698 9314 webserver.cc:460] Webserver started at http://127.0.0.1:42525/ using document root <none> and password file <none>
I20260924 21:20:54.581945 9314 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:54.582015 9314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:54.582165 9314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:54.583221 9314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5"
format_stamp: "Formatted at 2026-09-24 21:20:54 on dist-test-slave-cxpd"
server_key: "03e6c6ae792b1f3031f2db232f2f9a2d"
server_key_iv: "a682796314e5cd971932cb6907e6c60d"
server_key_version: "encryptionkey@0"
I20260924 21:20:54.583595 9314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5"
format_stamp: "Formatted at 2026-09-24 21:20:54 on dist-test-slave-cxpd"
server_key: "03e6c6ae792b1f3031f2db232f2f9a2d"
server_key_iv: "a682796314e5cd971932cb6907e6c60d"
server_key_version: "encryptionkey@0"
I20260924 21:20:54.584992 9314 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:20:54.585776 9329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:54.585956 9314 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:20:54.586064 9314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5"
format_stamp: "Formatted at 2026-09-24 21:20:54 on dist-test-slave-cxpd"
server_key: "03e6c6ae792b1f3031f2db232f2f9a2d"
server_key_iv: "a682796314e5cd971932cb6907e6c60d"
server_key_version: "encryptionkey@0"
I20260924 21:20:54.586138 9314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:54.599472 9314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:54.601714 9314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:54.601881 9314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:54.606168 9314 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35507
I20260924 21:20:54.606261 9381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35507 every 8 connection(s)
I20260924 21:20:54.606600 9314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:54.607481 9382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:54.609818 9382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5: Bootstrap starting.
I20260924 21:20:54.610383 9382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:54.610667 9382 log.cc:824] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:54.611394 9382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5: No bootstrap required, opened a new log
I20260924 21:20:54.613097 9382 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } }
I20260924 21:20:54.613235 9382 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:54.613260 9382 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed0a6f0b59c64aaa9abdbe7d70a109b5, State: Initialized, Role: FOLLOWER
I20260924 21:20:54.613411 9382 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } }
I20260924 21:20:54.613476 9382 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:54.613495 9382 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:54.613523 9382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:54.614007 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 9314
I20260924 21:20:54.614045 9382 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } }
I20260924 21:20:54.614123 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:54.614158 9382 leader_election.cc:302] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed0a6f0b59c64aaa9abdbe7d70a109b5; no voters:
I20260924 21:20:54.614355 20943 external_mini_cluster.cc:1428] Setting key 29ccec845301351a1bd8f1090505b007
I20260924 21:20:54.614359 9382 leader_election.cc:288] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:54.614384 9385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:54.614491 9385 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [term 1 LEADER]: Becoming Leader. State: Replica: ed0a6f0b59c64aaa9abdbe7d70a109b5, State: Running, Role: LEADER
I20260924 21:20:54.614632 9382 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:54.614625 9385 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } }
I20260924 21:20:54.615248 9386 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } } }
I20260924 21:20:54.615345 9386 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:54.615553 9387 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed0a6f0b59c64aaa9abdbe7d70a109b5. Latest consensus state: current_term: 1 leader_uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed0a6f0b59c64aaa9abdbe7d70a109b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35507 } } }
I20260924 21:20:54.615669 9387 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:54.615931 9396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:54.616518 9396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:54.618108 9396 catalog_manager.cc:1347] Generated new cluster ID: 7563767b03bd44d08433c5520917766b
I20260924 21:20:54.618172 9396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:20:54 dist-test-slave-cxpd krb5kdc[9301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 21:20:54.624759 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:54.614953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:44577 (local address 127.0.0.1:35507)
0924 21:20:54.615150 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:54.615160 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:54.615192 (+ 32us) server_negotiation.cc:407] Connection header received
0924 21:20:54.615312 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:54.615319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:54.615386 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:54.616266 (+ 880us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:54.616426 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:54.617705 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:54.618640 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:54.619056 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:54.619171 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:54.621531 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:54.621535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:54.621535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:54.621547 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:54.623206 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:54.623614 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:54.623616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:54.623617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:54.623725 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:54.623962 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:54.623963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:54.623963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:54.624195 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:54.624379 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:54.624463 (+ 84us) server_negotiation.cc:299] Negotiation successful
0924 21:20:54.624565 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":97,"threads_started":1}
I20260924 21:20:54.625778 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:54.633205 9390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:54.627119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:56343 (local address 127.0.0.1:35507)
0924 21:20:54.627154 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:54.627156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:54.627255 (+ 99us) server_negotiation.cc:407] Connection header received
0924 21:20:54.627323 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:54.627323 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:54.627335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:54.627389 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:54.627895 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:54.628366 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:54.629147 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:54.629374 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:54.629447 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:54.630600 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:54.630602 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:54.630602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:54.630609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:54.631969 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:54.632343 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:54.632344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:54.632344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:54.632395 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:54.632629 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:54.632630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:54.632631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:54.632742 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:54.632902 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:54.633012 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 21:20:54.633088 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:20:54.640576 9396 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:54.641497 9396 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:54.647876 9396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed0a6f0b59c64aaa9abdbe7d70a109b5: Generated new TSK 0
I20260924 21:20:54.648077 9396 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:54.656880 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 9314
W20260924 21:20:54.661223 9299 connection.cc:535] client connection to 127.0.0.1:35507 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35507): Connection reset by peer (error 104)
2026-09-24T21:20:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1326 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:54 dist-test-slave-cxpd krb5kdc[9413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:54 dist-test-slave-cxpd krb5kdc[9413](info): set up 2 sockets
Sep 24 21:20:54 dist-test-slave-cxpd krb5kdc[9413](info): commencing operation
krb5kdc: starting...
W20260924 21:20:55.712954 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:55 dist-test-slave-cxpd krb5kdc[9413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:55.827977 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:55.908329 9426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:55.908605 9426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:55.908646 9426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:55.909861 9426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:55.909942 9426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:55.909968 9426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:55.909997 9426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:55.910025 9426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:55.911540 9426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:55.911811 9426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:55.912042 9426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:55.916913 9434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:55.916942 9432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:55.916965 9431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:55.917284 9426 server_base.cc:1034] running on GCE node
I20260924 21:20:55.917541 9426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:55.917826 9426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:55.918972 9426 hybrid_clock.cc:648] HybridClock initialized: now 1790284855918958 us; error 29 us; skew 500 ppm
Sep 24 21:20:55 dist-test-slave-cxpd krb5kdc[9413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284855, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:55.921367 9426 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:55.921808 9426 webserver.cc:460] Webserver started at http://127.0.0.1:44891/ using document root <none> and password file <none>
I20260924 21:20:55.922036 9426 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:55.922112 9426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:55.922278 9426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:55.923317 9426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "53444a4fdb8d4a54b9344682818e8df4"
format_stamp: "Formatted at 2026-09-24 21:20:55 on dist-test-slave-cxpd"
server_key: "045834c650279219871edae436eb15fb"
server_key_iv: "3ab6181ac0b3fa0317fb594f24ff1b4d"
server_key_version: "encryptionkey@0"
I20260924 21:20:55.923683 9426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "53444a4fdb8d4a54b9344682818e8df4"
format_stamp: "Formatted at 2026-09-24 21:20:55 on dist-test-slave-cxpd"
server_key: "045834c650279219871edae436eb15fb"
server_key_iv: "3ab6181ac0b3fa0317fb594f24ff1b4d"
server_key_version: "encryptionkey@0"
I20260924 21:20:55.925108 9426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:20:55.925877 9441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:55.926046 9426 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:55.926146 9426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "53444a4fdb8d4a54b9344682818e8df4"
format_stamp: "Formatted at 2026-09-24 21:20:55 on dist-test-slave-cxpd"
server_key: "045834c650279219871edae436eb15fb"
server_key_iv: "3ab6181ac0b3fa0317fb594f24ff1b4d"
server_key_version: "encryptionkey@0"
I20260924 21:20:55.926229 9426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:55.941381 9426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:55.943500 9426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:55.943689 9426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:55.947769 9426 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38071
I20260924 21:20:55.947831 9493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38071 every 8 connection(s)
I20260924 21:20:55.948195 9426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:55.949043 9494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:55.951264 9494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4: Bootstrap starting.
I20260924 21:20:55.951790 9494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:55.952024 9494 log.cc:824] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:55.952781 9494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4: No bootstrap required, opened a new log
I20260924 21:20:55.952850 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 9426
I20260924 21:20:55.952965 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:55.953161 20943 external_mini_cluster.cc:1428] Setting key 2e721eec7a0db833ad34f0ce1cc13fd1
I20260924 21:20:55.954766 9494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53444a4fdb8d4a54b9344682818e8df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } }
I20260924 21:20:55.954933 9494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:55.954970 9494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53444a4fdb8d4a54b9344682818e8df4, State: Initialized, Role: FOLLOWER
I20260924 21:20:55.955185 9494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53444a4fdb8d4a54b9344682818e8df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } }
I20260924 21:20:55.955296 9494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:55.955343 9494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:55.955390 9494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:55.955917 9494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53444a4fdb8d4a54b9344682818e8df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } }
I20260924 21:20:55.956034 9494 leader_election.cc:302] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53444a4fdb8d4a54b9344682818e8df4; no voters:
I20260924 21:20:55.956229 9494 leader_election.cc:288] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:55.956254 9499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:55.956360 9499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [term 1 LEADER]: Becoming Leader. State: Replica: 53444a4fdb8d4a54b9344682818e8df4, State: Running, Role: LEADER
I20260924 21:20:55.956487 9494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:55.956498 9499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53444a4fdb8d4a54b9344682818e8df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } }
I20260924 21:20:55.956991 9500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53444a4fdb8d4a54b9344682818e8df4. Latest consensus state: current_term: 1 leader_uuid: "53444a4fdb8d4a54b9344682818e8df4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53444a4fdb8d4a54b9344682818e8df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } } }
I20260924 21:20:55.957084 9500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:55.957285 9501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53444a4fdb8d4a54b9344682818e8df4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53444a4fdb8d4a54b9344682818e8df4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38071 } } }
I20260924 21:20:55.957401 9501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:55.957585 9506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:55.958179 9506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:55.960011 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:55.953690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:49805 (local address 127.0.0.1:38071)
0924 21:20:55.953855 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:55.953864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:55.953899 (+ 35us) server_negotiation.cc:407] Connection header received
0924 21:20:55.954752 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:55.954759 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:55.954809 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:55.954920 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:55.955210 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:55.956600 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:55.957750 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:55.958361 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:55.958910 (+ 549us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:55.959342 (+ 432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:55.959345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:55.959346 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:55.959540 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:55.959578 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:55.959592 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:55.959681 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:55.959802 (+ 121us) server_negotiation.cc:299] Negotiation successful
0924 21:20:55.959831 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260924 21:20:55.960212 9506 catalog_manager.cc:1347] Generated new cluster ID: 6808b2b217154a4cac768f5ae17d8509
I20260924 21:20:55.960266 9506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:55.960929 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:55.971773 9498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:55.965131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:53263 (local address 127.0.0.1:38071)
0924 21:20:55.965168 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:55.965170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:55.968996 (+ 3826us) server_negotiation.cc:407] Connection header received
0924 21:20:55.969076 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:55.969077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:55.969089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:55.969158 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:55.969310 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:55.969790 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:55.970528 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:55.970740 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:55.970823 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:55.971501 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:55.971503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:55.971503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:55.971508 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:55.971521 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:55.971525 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:55.971578 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:55.971659 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 21:20:55.971672 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:20:55.985407 9506 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:55.986097 9506 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:56.002803 9506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53444a4fdb8d4a54b9344682818e8df4: Generated new TSK 0
I20260924 21:20:56.003015 9506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:56.010376 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 9426
W20260924 21:20:56.014748 9411 connection.cc:535] client connection to 127.0.0.1:38071 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38071): Connection reset by peer (error 104)
2026-09-24T21:20:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1353 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:56 dist-test-slave-cxpd krb5kdc[9525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:56 dist-test-slave-cxpd krb5kdc[9525](info): set up 2 sockets
Sep 24 21:20:56 dist-test-slave-cxpd krb5kdc[9525](info): commencing operation
krb5kdc: starting...
W20260924 21:20:57.044981 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:57 dist-test-slave-cxpd krb5kdc[9525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284857, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:57.167724 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33797
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33797
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:57.254760 9538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:57.255026 9538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:57.255080 9538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:57.256305 9538 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:57.256369 9538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:57.256393 9538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:57.256414 9538 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:57.256443 9538 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:57.257884 9538 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9538
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:57.258159 9538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:57.258373 9538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:57.263401 9544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:57.263509 9538 server_base.cc:1034] running on GCE node
W20260924 21:20:57.263413 9546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:57.263413 9543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:57.263926 9538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:57.264258 9538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:57.265439 9538 hybrid_clock.cc:648] HybridClock initialized: now 1790284857265413 us; error 35 us; skew 500 ppm
Sep 24 21:20:57 dist-test-slave-cxpd krb5kdc[9525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284857, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:57.267971 9538 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:57.268505 9538 webserver.cc:460] Webserver started at http://127.0.0.1:37277/ using document root <none> and password file <none>
I20260924 21:20:57.268765 9538 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:57.268898 9538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:57.269075 9538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:57.270226 9538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "0622024a7883469299d420e2246e5a04"
format_stamp: "Formatted at 2026-09-24 21:20:57 on dist-test-slave-cxpd"
server_key: "5678b932226d390fde81b7c08e00c983"
server_key_iv: "9220a63cc1e50f73e0f8a765fef0ee57"
server_key_version: "encryptionkey@0"
I20260924 21:20:57.270615 9538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "0622024a7883469299d420e2246e5a04"
format_stamp: "Formatted at 2026-09-24 21:20:57 on dist-test-slave-cxpd"
server_key: "5678b932226d390fde81b7c08e00c983"
server_key_iv: "9220a63cc1e50f73e0f8a765fef0ee57"
server_key_version: "encryptionkey@0"
I20260924 21:20:57.272085 9538 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:20:57.272984 9553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:57.273160 9538 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:20:57.273281 9538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "0622024a7883469299d420e2246e5a04"
format_stamp: "Formatted at 2026-09-24 21:20:57 on dist-test-slave-cxpd"
server_key: "5678b932226d390fde81b7c08e00c983"
server_key_iv: "9220a63cc1e50f73e0f8a765fef0ee57"
server_key_version: "encryptionkey@0"
I20260924 21:20:57.273353 9538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:57.276765 9538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:57.279377 9538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:57.279532 9538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:57.283392 9538 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33797
I20260924 21:20:57.283422 9605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33797 every 8 connection(s)
I20260924 21:20:57.283740 9538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:57.284598 9606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:57.286844 9606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04: Bootstrap starting.
I20260924 21:20:57.287369 9606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:57.287602 9606 log.cc:824] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:57.288286 9606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04: No bootstrap required, opened a new log
I20260924 21:20:57.289908 9606 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0622024a7883469299d420e2246e5a04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260924 21:20:57.290045 9606 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:57.290076 9606 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0622024a7883469299d420e2246e5a04, State: Initialized, Role: FOLLOWER
I20260924 21:20:57.290210 9606 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0622024a7883469299d420e2246e5a04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260924 21:20:57.290290 9606 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:57.290313 9606 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:57.290341 9606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:57.290818 9606 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0622024a7883469299d420e2246e5a04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260924 21:20:57.290901 9606 leader_election.cc:302] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0622024a7883469299d420e2246e5a04; no voters:
I20260924 21:20:57.291020 9606 leader_election.cc:288] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:57.291100 9609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:57.291275 9609 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [term 1 LEADER]: Becoming Leader. State: Replica: 0622024a7883469299d420e2246e5a04, State: Running, Role: LEADER
I20260924 21:20:57.291316 9606 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:57.291368 9609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0622024a7883469299d420e2246e5a04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } }
I20260924 21:20:57.291922 9610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0622024a7883469299d420e2246e5a04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0622024a7883469299d420e2246e5a04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } } }
I20260924 21:20:57.292119 9610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:57.292047 9611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0622024a7883469299d420e2246e5a04. Latest consensus state: current_term: 1 leader_uuid: "0622024a7883469299d420e2246e5a04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0622024a7883469299d420e2246e5a04" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33797 } } }
I20260924 21:20:57.292261 9611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:57.292443 9618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:57.292863 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 9538
I20260924 21:20:57.292984 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:57.293008 9618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:57.293191 20943 external_mini_cluster.cc:1428] Setting key 7c52931808471325f4ab9deaa42ae3a9
I20260924 21:20:57.294821 9618 catalog_manager.cc:1347] Generated new cluster ID: 90f3eaaefc3747e588496fa76ec1d1ef
I20260924 21:20:57.294878 9618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:57.295650 9628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:57.293634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:60197 (local address 127.0.0.1:33797)
0924 21:20:57.294155 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:57.294162 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:57.294186 (+ 24us) server_negotiation.cc:407] Connection header received
0924 21:20:57.294312 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:57.294319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:57.294352 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:57.294502 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:57.295209 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:57.295212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:57.295213 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:57.295217 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:57.295263 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:57.295274 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:57.295338 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:57.295418 (+ 80us) server_negotiation.cc:299] Negotiation successful
0924 21:20:57.295464 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260924 21:20:57.296525 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:57.299022 9628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:57.297929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:50321 (local address 127.0.0.1:33797)
0924 21:20:57.297967 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:57.297969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:57.298093 (+ 124us) server_negotiation.cc:407] Connection header received
0924 21:20:57.298167 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:57.298168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:57.298173 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:57.298217 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:57.298787 (+ 570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:57.298789 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:57.298789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:57.298794 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:57.298809 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:57.298813 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:57.298869 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:57.298926 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 21:20:57.298942 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:20:57.326726 9618 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:57.326931 9618 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:57.336503 9618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0622024a7883469299d420e2246e5a04: Generated new TSK 0
I20260924 21:20:57.336723 9618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:57.345026 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 9538
2026-09-24T21:20:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1337 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:57 dist-test-slave-cxpd krb5kdc[9637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:57 dist-test-slave-cxpd krb5kdc[9637](info): set up 2 sockets
Sep 24 21:20:57 dist-test-slave-cxpd krb5kdc[9637](info): commencing operation
krb5kdc: starting...
W20260924 21:20:58.392819 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:58 dist-test-slave-cxpd krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:58.511322 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40861
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:58.590153 9650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:58.590366 9650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:58.590402 9650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:58.591530 9650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:58.591601 9650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:58.591627 9650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:58.591655 9650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:58.591678 9650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:58.593209 9650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:58.593468 9650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:58.593685 9650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:58.598358 9656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:58.598464 9658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:58.598573 9650 server_base.cc:1034] running on GCE node
W20260924 21:20:58.598366 9655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:58.598850 9650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:58.599123 9650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:58.600291 9650 hybrid_clock.cc:648] HybridClock initialized: now 1790284858600280 us; error 41 us; skew 500 ppm
Sep 24 21:20:58 dist-test-slave-cxpd krb5kdc[9637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284858, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:58.607699 9650 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:58.608129 9650 webserver.cc:460] Webserver started at http://127.0.0.1:36471/ using document root <none> and password file <none>
I20260924 21:20:58.608311 9650 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:58.608350 9650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:58.608435 9650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:58.609469 9650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "e6b7911e457e45a19501e13604c886cb"
format_stamp: "Formatted at 2026-09-24 21:20:58 on dist-test-slave-cxpd"
server_key: "2772d9f2f531b217d116ec57d04f1861"
server_key_iv: "02368956c7a5fb1997ab5800ba14508e"
server_key_version: "encryptionkey@0"
I20260924 21:20:58.609761 9650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "e6b7911e457e45a19501e13604c886cb"
format_stamp: "Formatted at 2026-09-24 21:20:58 on dist-test-slave-cxpd"
server_key: "2772d9f2f531b217d116ec57d04f1861"
server_key_iv: "02368956c7a5fb1997ab5800ba14508e"
server_key_version: "encryptionkey@0"
I20260924 21:20:58.610972 9650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:20:58.611615 9665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:58.611790 9650 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:20:58.611871 9650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "e6b7911e457e45a19501e13604c886cb"
format_stamp: "Formatted at 2026-09-24 21:20:58 on dist-test-slave-cxpd"
server_key: "2772d9f2f531b217d116ec57d04f1861"
server_key_iv: "02368956c7a5fb1997ab5800ba14508e"
server_key_version: "encryptionkey@0"
I20260924 21:20:58.611922 9650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:58.619601 9650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:58.621407 9650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:58.621515 9650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:58.625308 9650 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40861
I20260924 21:20:58.625352 9717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40861 every 8 connection(s)
I20260924 21:20:58.625684 9650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:58.626514 9718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:58.628685 9718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb: Bootstrap starting.
I20260924 21:20:58.629294 9718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:58.629568 9718 log.cc:824] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:58.630214 9718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb: No bootstrap required, opened a new log
I20260924 21:20:58.631726 9718 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b7911e457e45a19501e13604c886cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } }
I20260924 21:20:58.631872 9718 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:58.631917 9718 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b7911e457e45a19501e13604c886cb, State: Initialized, Role: FOLLOWER
I20260924 21:20:58.632071 9718 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b7911e457e45a19501e13604c886cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } }
I20260924 21:20:58.632165 9718 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:58.632210 9718 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:58.632257 9718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:58.632741 9718 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b7911e457e45a19501e13604c886cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } }
I20260924 21:20:58.632869 9718 leader_election.cc:302] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6b7911e457e45a19501e13604c886cb; no voters:
I20260924 21:20:58.633036 9718 leader_election.cc:288] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:58.633065 9721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:58.633235 9721 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [term 1 LEADER]: Becoming Leader. State: Replica: e6b7911e457e45a19501e13604c886cb, State: Running, Role: LEADER
I20260924 21:20:58.633279 9718 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:58.633349 9721 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b7911e457e45a19501e13604c886cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } }
I20260924 21:20:58.633754 9723 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6b7911e457e45a19501e13604c886cb. Latest consensus state: current_term: 1 leader_uuid: "e6b7911e457e45a19501e13604c886cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b7911e457e45a19501e13604c886cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } } }
I20260924 21:20:58.633874 9722 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6b7911e457e45a19501e13604c886cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b7911e457e45a19501e13604c886cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40861 } } }
I20260924 21:20:58.633891 9723 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:58.633965 9722 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:58.634286 9730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:58.634707 9730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:58.635917 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 9650
I20260924 21:20:58.636031 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:58.636250 9730 catalog_manager.cc:1347] Generated new cluster ID: 87d4b925621743a09a5e58a668072692
I20260924 21:20:58.636276 20943 external_mini_cluster.cc:1428] Setting key 0d58f3d8df1b983dfb3cc67dfa65324b
I20260924 21:20:58.636299 9730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:20:58 dist-test-slave-cxpd krb5kdc[9637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260924 21:20:58.647444 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:58.636698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:55923 (local address 127.0.0.1:40861)
0924 21:20:58.636851 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:58.636859 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:58.637052 (+ 193us) server_negotiation.cc:407] Connection header received
0924 21:20:58.637170 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:58.637177 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:58.637233 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:58.637371 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:58.638044 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:58.639447 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:58.640476 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:58.641155 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:58.641292 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:58.643949 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:58.643953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:58.643953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:58.643965 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:58.645846 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:58.646279 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:58.646281 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:58.646282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:58.646377 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:58.646603 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:58.646604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:58.646604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:58.646874 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:58.647005 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:58.647140 (+ 135us) server_negotiation.cc:299] Negotiation successful
0924 21:20:58.647231 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":60,"threads_started":1}
I20260924 21:20:58.648583 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:58.659991 9730 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:58.660487 9730 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:58.666924 9741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:58.649999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:50009 (local address 127.0.0.1:40861)
0924 21:20:58.650029 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:58.650030 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:58.659770 (+ 9740us) server_negotiation.cc:407] Connection header received
0924 21:20:58.659864 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:58.659865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:58.659876 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:58.659927 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:58.660653 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:58.661386 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:58.662399 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:58.662703 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:58.662761 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:58.664162 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:58.664164 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:58.664164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:20:58.664170 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:58.665717 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:58.666109 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:58.666110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:58.666111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:58.666176 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:20:58.666404 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:20:58.666404 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:20:58.666405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:20:58.666533 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:58.666648 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:58.666767 (+ 119us) server_negotiation.cc:299] Negotiation successful
0924 21:20:58.666833 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260924 21:20:58.676256 9730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6b7911e457e45a19501e13604c886cb: Generated new TSK 0
I20260924 21:20:58.676487 9730 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:20:58.680154 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 9650
W20260924 21:20:58.685075 9635 connection.cc:535] client connection to 127.0.0.1:40861 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40861): Connection reset by peer (error 104)
2026-09-24T21:20:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:20:58 dist-test-slave-cxpd krb5kdc[9749](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:20:58 dist-test-slave-cxpd krb5kdc[9749](info): set up 2 sockets
Sep 24 21:20:58 dist-test-slave-cxpd krb5kdc[9749](info): commencing operation
krb5kdc: starting...
W20260924 21:20:59.716954 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:20:59 dist-test-slave-cxpd krb5kdc[9749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:20:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:20:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:20:59.835366 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41377
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:20:59.914438 9762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:20:59.914670 9762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:20:59.914708 9762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:20:59.915848 9762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:20:59.915916 9762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:20:59.915940 9762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:20:59.915963 9762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:20:59.915989 9762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:20:59.917428 9762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41377
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:20:59.917689 9762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:20:59.917893 9762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:20:59.922585 9770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:20:59.922592 9768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:59.922744 9762 server_base.cc:1034] running on GCE node
W20260924 21:20:59.922837 9767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:20:59.923108 9762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:20:59.923421 9762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:20:59.924597 9762 hybrid_clock.cc:648] HybridClock initialized: now 1790284859924582 us; error 37 us; skew 500 ppm
Sep 24 21:20:59 dist-test-slave-cxpd krb5kdc[9749](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284859, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:20:59.927155 9762 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:20:59.927551 9762 webserver.cc:460] Webserver started at http://127.0.0.1:34699/ using document root <none> and password file <none>
I20260924 21:20:59.927765 9762 fs_manager.cc:359] Metadata directory not provided
I20260924 21:20:59.927829 9762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:20:59.927975 9762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:20:59.929107 9762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "4322c28d91e54a06b292163a71a892e2"
format_stamp: "Formatted at 2026-09-24 21:20:59 on dist-test-slave-cxpd"
server_key: "777137e226279142d47b2f709ed2b32a"
server_key_iv: "117e92ec8a2603bc828f8dbe0b9781a5"
server_key_version: "encryptionkey@0"
I20260924 21:20:59.929467 9762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "4322c28d91e54a06b292163a71a892e2"
format_stamp: "Formatted at 2026-09-24 21:20:59 on dist-test-slave-cxpd"
server_key: "777137e226279142d47b2f709ed2b32a"
server_key_iv: "117e92ec8a2603bc828f8dbe0b9781a5"
server_key_version: "encryptionkey@0"
I20260924 21:20:59.930788 9762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:20:59.931569 9777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:20:59.931740 9762 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:20:59.931835 9762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "4322c28d91e54a06b292163a71a892e2"
format_stamp: "Formatted at 2026-09-24 21:20:59 on dist-test-slave-cxpd"
server_key: "777137e226279142d47b2f709ed2b32a"
server_key_iv: "117e92ec8a2603bc828f8dbe0b9781a5"
server_key_version: "encryptionkey@0"
I20260924 21:20:59.931910 9762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:20:59.957106 9762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:20:59.959622 9762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:20:59.959785 9762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:20:59.963771 9762 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41377
I20260924 21:20:59.963814 9829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41377 every 8 connection(s)
I20260924 21:20:59.964181 9762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:20:59.965081 9830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:20:59.967307 9830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2: Bootstrap starting.
I20260924 21:20:59.967870 9830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2: Neither blocks nor log segments found. Creating new log.
I20260924 21:20:59.968144 9830 log.cc:824] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2: Log is configured to *not* fsync() on all Append() calls
I20260924 21:20:59.968834 9830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2: No bootstrap required, opened a new log
I20260924 21:20:59.970276 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 9762
I20260924 21:20:59.970412 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:20:59.970409 9830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4322c28d91e54a06b292163a71a892e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41377 } }
I20260924 21:20:59.970575 9830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:20:59.970621 20943 external_mini_cluster.cc:1428] Setting key 5d5b1dc80c0dbb68fe51055ab4f89900
I20260924 21:20:59.970602 9830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4322c28d91e54a06b292163a71a892e2, State: Initialized, Role: FOLLOWER
I20260924 21:20:59.970777 9830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4322c28d91e54a06b292163a71a892e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41377 } }
I20260924 21:20:59.970868 9830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:20:59.970904 9830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:20:59.970959 9830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:20:59.971980 9830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4322c28d91e54a06b292163a71a892e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41377 } }
I20260924 21:20:59.972081 9830 leader_election.cc:302] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4322c28d91e54a06b292163a71a892e2; no voters:
I20260924 21:20:59.972229 9830 leader_election.cc:288] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:20:59.972281 9835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:20:59.972421 9835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [term 1 LEADER]: Becoming Leader. State: Replica: 4322c28d91e54a06b292163a71a892e2, State: Running, Role: LEADER
I20260924 21:20:59.972455 9830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:20:59.972504 9835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4322c28d91e54a06b292163a71a892e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41377 } }
I20260924 21:20:59.972832 9836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4322c28d91e54a06b292163a71a892e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4322c28d91e54a06b292163a71a892e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41377 } } }
I20260924 21:20:59.972960 9836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:59.972818 9835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4322c28d91e54a06b292163a71a892e2. Latest consensus state: current_term: 1 leader_uuid: "4322c28d91e54a06b292163a71a892e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4322c28d91e54a06b292163a71a892e2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41377 } } }
I20260924 21:20:59.973143 9835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2 [sys.catalog]: This master's current role is: LEADER
I20260924 21:20:59.973268 9844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:20:59.973891 9844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:20:59.975666 9844 catalog_manager.cc:1347] Generated new cluster ID: 47994f2028e54982815628f79a867b6d
I20260924 21:20:59.975719 9844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:20:59.976469 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:59.971066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:44827 (local address 127.0.0.1:41377)
0924 21:20:59.971286 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:59.971296 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:59.971330 (+ 34us) server_negotiation.cc:407] Connection header received
0924 21:20:59.971455 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:59.971462 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:59.971530 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:59.971716 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:59.971858 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:59.973263 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:59.974434 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:59.975055 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:59.975161 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:59.975855 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:59.975859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:59.975860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:59.976029 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:59.976058 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:59.976068 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:59.976159 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:59.976240 (+ 81us) server_negotiation.cc:299] Negotiation successful
0924 21:20:59.976267 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260924 21:20:59.977339 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:20:59.987931 9834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:20:59.978624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:53149 (local address 127.0.0.1:41377)
0924 21:20:59.978660 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:20:59.978662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:20:59.984988 (+ 6326us) server_negotiation.cc:407] Connection header received
0924 21:20:59.985133 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:20:59.985134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:20:59.985145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:20:59.985294 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:20:59.985446 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:59.985932 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:59.986683 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:20:59.986912 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:20:59.986985 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:20:59.987686 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:20:59.987688 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:20:59.987688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:20:59.987693 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:20:59.987706 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:20:59.987710 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:20:59.987763 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0924 21:20:59.987822 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 21:20:59.987834 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:20:59.995218 9844 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:20:59.995721 9844 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:20:59.997834 9844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4322c28d91e54a06b292163a71a892e2: Generated new TSK 0
I20260924 21:20:59.998011 9844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:21:00.003770 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 9762
W20260924 21:21:00.007848 9747 connection.cc:535] client connection to 127.0.0.1:41377 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41377): Connection reset by peer (error 104)
2026-09-24T21:21:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1322 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:21:00 dist-test-slave-cxpd krb5kdc[9861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:21:00 dist-test-slave-cxpd krb5kdc[9861](info): set up 2 sockets
Sep 24 21:21:00 dist-test-slave-cxpd krb5kdc[9861](info): commencing operation
krb5kdc: starting...
W20260924 21:21:01.033152 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:21:01 dist-test-slave-cxpd krb5kdc[9861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284861, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:21:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:21:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260924 21:21:01.151688 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42239
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:21:01.230614 9874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:01.230829 9874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:01.230852 9874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:01.232895 9874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:01.232961 9874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:01.232977 9874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:01.232993 9874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:01.233016 9874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:01.234622 9874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42239
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:01.234879 9874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:01.235113 9874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:01.240126 9879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:01.240229 9880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:01.240235 9874 server_base.cc:1034] running on GCE node
W20260924 21:21:01.240105 9882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:01.240582 9874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:01.240931 9874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:01.242110 9874 hybrid_clock.cc:648] HybridClock initialized: now 1790284861242100 us; error 30 us; skew 500 ppm
Sep 24 21:21:01 dist-test-slave-cxpd krb5kdc[9861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284861, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:01.244392 9874 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260924 21:21:01.244752 9874 webserver.cc:460] Webserver started at http://127.0.0.1:40877/ using document root <none> and password file <none>
I20260924 21:21:01.244984 9874 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:01.245029 9874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:01.245121 9874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:01.246093 9874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "d08f86be8f0d455ab6532230a5dbee9d"
format_stamp: "Formatted at 2026-09-24 21:21:01 on dist-test-slave-cxpd"
server_key: "e55521e6b96c780e03afc18a208038f0"
server_key_iv: "6b72de1dbd4bbf07e91659aed3608f13"
server_key_version: "encryptionkey@0"
I20260924 21:21:01.246373 9874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "d08f86be8f0d455ab6532230a5dbee9d"
format_stamp: "Formatted at 2026-09-24 21:21:01 on dist-test-slave-cxpd"
server_key: "e55521e6b96c780e03afc18a208038f0"
server_key_iv: "6b72de1dbd4bbf07e91659aed3608f13"
server_key_version: "encryptionkey@0"
I20260924 21:21:01.247560 9874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260924 21:21:01.248260 9889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:01.248435 9874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:21:01.248519 9874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "d08f86be8f0d455ab6532230a5dbee9d"
format_stamp: "Formatted at 2026-09-24 21:21:01 on dist-test-slave-cxpd"
server_key: "e55521e6b96c780e03afc18a208038f0"
server_key_iv: "6b72de1dbd4bbf07e91659aed3608f13"
server_key_version: "encryptionkey@0"
I20260924 21:21:01.248587 9874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:01.251652 9874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:01.253779 9874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:01.253927 9874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:01.257838 9874 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42239
I20260924 21:21:01.257887 9941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42239 every 8 connection(s)
I20260924 21:21:01.258167 9874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:21:01.259011 9942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:01.261173 9942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d: Bootstrap starting.
I20260924 21:21:01.261652 9942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:01.261873 9942 log.cc:824] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:01.262460 9942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d: No bootstrap required, opened a new log
I20260924 21:21:01.263947 9942 raft_consensus.cc:348] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08f86be8f0d455ab6532230a5dbee9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42239 } }
I20260924 21:21:01.264082 9942 raft_consensus.cc:374] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:21:01.264106 9942 raft_consensus.cc:729] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d08f86be8f0d455ab6532230a5dbee9d, State: Initialized, Role: FOLLOWER
I20260924 21:21:01.264235 9942 consensus_queue.cc:260] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08f86be8f0d455ab6532230a5dbee9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42239 } }
I20260924 21:21:01.264310 9942 raft_consensus.cc:388] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:21:01.264333 9942 raft_consensus.cc:482] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:21:01.264366 9942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:01.264914 9942 raft_consensus.cc:504] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08f86be8f0d455ab6532230a5dbee9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42239 } }
I20260924 21:21:01.264997 9942 leader_election.cc:302] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d08f86be8f0d455ab6532230a5dbee9d; no voters:
I20260924 21:21:01.265122 9942 leader_election.cc:288] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:21:01.265158 9945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:21:01.265295 9945 raft_consensus.cc:686] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [term 1 LEADER]: Becoming Leader. State: Replica: d08f86be8f0d455ab6532230a5dbee9d, State: Running, Role: LEADER
I20260924 21:21:01.265374 9942 sys_catalog.cc:564] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:01.265419 9945 consensus_queue.cc:237] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08f86be8f0d455ab6532230a5dbee9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42239 } }
I20260924 21:21:01.265779 9947 sys_catalog.cc:455] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d08f86be8f0d455ab6532230a5dbee9d. Latest consensus state: current_term: 1 leader_uuid: "d08f86be8f0d455ab6532230a5dbee9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08f86be8f0d455ab6532230a5dbee9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42239 } } }
I20260924 21:21:01.265913 9947 sys_catalog.cc:458] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:01.265899 9946 sys_catalog.cc:455] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d08f86be8f0d455ab6532230a5dbee9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d08f86be8f0d455ab6532230a5dbee9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42239 } } }
I20260924 21:21:01.265990 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 9874
I20260924 21:21:01.266064 9946 sys_catalog.cc:458] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:01.266103 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:21:01.266322 20943 external_mini_cluster.cc:1428] Setting key cf7f0bcc934652242985eba00aaa12da
W20260924 21:21:01.267705 9963 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:21:01.267776 9963 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:21:01.267836 9950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:21:01.268040 9950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:21:01.268724 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:01.266759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:33027 (local address 127.0.0.1:42239)
0924 21:21:01.266918 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:01.266928 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:01.267005 (+ 77us) server_negotiation.cc:407] Connection header received
0924 21:21:01.267151 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:01.267160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:01.267196 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:01.267452 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:01.268132 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:01.268136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:01.268136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:01.268141 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:01.268180 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:01.268193 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:01.268454 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:01.268483 (+ 29us) server_negotiation.cc:299] Negotiation successful
0924 21:21:01.268524 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260924 21:21:01.269616 9950 catalog_manager.cc:1347] Generated new cluster ID: f9bf84c73b854f19b6cd7590edb104ae
I20260924 21:21:01.269622 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:21:01.269665 9950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:21:01.273761 9957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:01.270943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.132:37069 (local address 127.0.0.1:42239)
0924 21:21:01.272864 (+ 1921us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:01.272866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:01.272875 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:21:01.272909 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:01.272910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:01.272913 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:01.272981 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:01.273550 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:01.273551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:01.273552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:01.273556 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:01.273570 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:01.273574 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:01.273619 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:01.273683 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 21:21:01.273698 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1900}
I20260924 21:21:01.281308 9950 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:21:01.281451 9950 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:21:01.298805 9950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d08f86be8f0d455ab6532230a5dbee9d: Generated new TSK 0
I20260924 21:21:01.299288 9950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:21:01.303972 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 9874
2026-09-24T21:21:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1300 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:21:01 dist-test-slave-cxpd krb5kdc[9973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:21:01 dist-test-slave-cxpd krb5kdc[9973](info): set up 2 sockets
Sep 24 21:21:01 dist-test-slave-cxpd krb5kdc[9973](info): commencing operation
krb5kdc: starting...
W20260924 21:21:02.328917 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.007s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:21:02 dist-test-slave-cxpd krb5kdc[9973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:21:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:21:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:21:02.447548 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:42633
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:42633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:21:02.528316 9986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:02.528566 9986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:02.528604 9986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:02.529778 9986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:02.529851 9986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:02.529879 9986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:02.529910 9986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:02.529942 9986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:02.531481 9986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:42633
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:42633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:02.531711 9986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:02.531898 9986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:21:02.536749 9986 server_base.cc:1034] running on GCE node
W20260924 21:21:02.536674 9992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:02.536681 9994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:02.536674 9991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:02.537191 9986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:02.537490 9986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:02.538690 9986 hybrid_clock.cc:648] HybridClock initialized: now 1790284862538682 us; error 34 us; skew 500 ppm
Sep 24 21:21:02 dist-test-slave-cxpd krb5kdc[9973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284862, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:02.541255 9986 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:21:02.541746 9986 webserver.cc:460] Webserver started at http://127.20.115.254:33715/ using document root <none> and password file <none>
I20260924 21:21:02.541945 9986 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:02.541986 9986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:02.542073 9986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:02.543071 9986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "3f0f89fac28b4e539182af50e6401da5"
format_stamp: "Formatted at 2026-09-24 21:21:02 on dist-test-slave-cxpd"
server_key: "a167f8611b9802fbdf7e2bca6a02b434"
server_key_iv: "52ffb55e214473cf177ff5653932ae36"
server_key_version: "encryptionkey@0"
I20260924 21:21:02.543391 9986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "3f0f89fac28b4e539182af50e6401da5"
format_stamp: "Formatted at 2026-09-24 21:21:02 on dist-test-slave-cxpd"
server_key: "a167f8611b9802fbdf7e2bca6a02b434"
server_key_iv: "52ffb55e214473cf177ff5653932ae36"
server_key_version: "encryptionkey@0"
I20260924 21:21:02.544613 9986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:21:02.545372 10001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:02.545519 9986 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:02.545604 9986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "3f0f89fac28b4e539182af50e6401da5"
format_stamp: "Formatted at 2026-09-24 21:21:02 on dist-test-slave-cxpd"
server_key: "a167f8611b9802fbdf7e2bca6a02b434"
server_key_iv: "52ffb55e214473cf177ff5653932ae36"
server_key_version: "encryptionkey@0"
I20260924 21:21:02.545692 9986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:02.567859 9986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:02.569820 9986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:02.569988 9986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:02.573860 9986 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:42633
I20260924 21:21:02.573908 10053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:42633 every 8 connection(s)
I20260924 21:21:02.574282 9986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:21:02.575158 10054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:02.577430 10054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5: Bootstrap starting.
I20260924 21:21:02.577952 10054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:02.578181 10054 log.cc:824] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:02.578812 10054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5: No bootstrap required, opened a new log
I20260924 21:21:02.580307 10054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f0f89fac28b4e539182af50e6401da5" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42633 } }
I20260924 21:21:02.580447 10054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:21:02.580469 10054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0f89fac28b4e539182af50e6401da5, State: Initialized, Role: FOLLOWER
I20260924 21:21:02.580608 10054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f0f89fac28b4e539182af50e6401da5" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42633 } }
I20260924 21:21:02.580699 10054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:21:02.580724 10054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:21:02.580756 10054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:02.581629 10054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f0f89fac28b4e539182af50e6401da5" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42633 } }
I20260924 21:21:02.581727 10054 leader_election.cc:302] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f0f89fac28b4e539182af50e6401da5; no voters:
I20260924 21:21:02.581851 10054 leader_election.cc:288] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:21:02.581919 10057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:21:02.582083 10057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [term 1 LEADER]: Becoming Leader. State: Replica: 3f0f89fac28b4e539182af50e6401da5, State: Running, Role: LEADER
I20260924 21:21:02.582129 10054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:02.582326 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 9986
I20260924 21:21:02.582485 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:21:02.582717 20943 external_mini_cluster.cc:1428] Setting key 8b4dd24b31b228d1f55401e040289e1e
I20260924 21:21:02.582749 10057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f0f89fac28b4e539182af50e6401da5" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42633 } }
I20260924 21:21:02.583350 10058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f0f89fac28b4e539182af50e6401da5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f0f89fac28b4e539182af50e6401da5" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42633 } } }
I20260924 21:21:02.583452 10058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:02.583674 10073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:21:02.583859 10059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f0f89fac28b4e539182af50e6401da5. Latest consensus state: current_term: 1 leader_uuid: "3f0f89fac28b4e539182af50e6401da5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f0f89fac28b4e539182af50e6401da5" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 42633 } } }
I20260924 21:21:02.583926 10059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5 [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:02.584218 10073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:21:02.585796 10073 catalog_manager.cc:1347] Generated new cluster ID: f929f978cfa143aaaeb331f16f9f8e7d
I20260924 21:21:02.585848 10073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 24 21:21:02 dist-test-slave-cxpd krb5kdc[9973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:21:02.593359 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:02.583685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48241 (local address 127.20.115.254:42633)
0924 21:21:02.584194 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:02.584202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:02.584227 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:21:02.584355 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:02.584363 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:02.584411 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:02.584545 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:02.585301 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:02.586381 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:02.587120 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:02.587549 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:02.587681 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:02.590098 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:02.590101 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:02.590102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:02.590145 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:02.591736 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:02.592114 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:02.592118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:02.592118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:02.592239 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:02.592462 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:02.592462 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:02.592463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:02.592692 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:02.592898 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:02.592985 (+ 87us) server_negotiation.cc:299] Negotiation successful
0924 21:21:02.593087 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":69,"threads_started":1}
I20260924 21:21:02.594236 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:21:02.601483 10074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:02.595566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48877 (local address 127.20.115.254:42633)
0924 21:21:02.595606 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:02.595608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:02.595729 (+ 121us) server_negotiation.cc:407] Connection header received
0924 21:21:02.595800 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:02.595801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:02.595811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:02.595865 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:02.596358 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:02.596906 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:02.597596 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:02.597829 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:02.597910 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:02.599031 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:02.599032 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:02.599032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:02.599039 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:02.600365 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:02.600716 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:02.600717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:02.600718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:02.600769 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:02.600988 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:02.600988 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:02.600989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:02.601096 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:02.601203 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:02.601315 (+ 112us) server_negotiation.cc:299] Negotiation successful
0924 21:21:02.601373 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260924 21:21:02.608840 10073 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:21:02.609470 10073 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:21:02.613919 10073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f0f89fac28b4e539182af50e6401da5: Generated new TSK 0
I20260924 21:21:02.614084 10073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:21:02.617364 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 9986
W20260924 21:21:02.621191 9971 connection.cc:535] client connection to 127.20.115.254:42633 recv error: Network error: failed to read from TLS socket (remote: 127.20.115.254:42633): Connection reset by peer (error 104)
2026-09-24T21:21:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1312 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:21:02 dist-test-slave-cxpd krb5kdc[10085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:21:02 dist-test-slave-cxpd krb5kdc[10085](info): set up 2 sockets
Sep 24 21:21:02 dist-test-slave-cxpd krb5kdc[10085](info): commencing operation
krb5kdc: starting...
W20260924 21:21:03.648931 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:21:03 dist-test-slave-cxpd krb5kdc[10085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:21:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:21:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:21:03.768023 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:37825
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:37825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:21:03.848116 10098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:03.848322 10098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:03.848344 10098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:03.849516 10098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:03.849561 10098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:03.849576 10098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:03.849588 10098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:03.849601 10098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:03.851080 10098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:37825
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:37825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:03.851325 10098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:03.851545 10098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:21:03.856760 10098 server_base.cc:1034] running on GCE node
W20260924 21:21:03.856732 10104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:03.856732 10103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:03.856771 10106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:03.857272 10098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:03.857561 10098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:03.858741 10098 hybrid_clock.cc:648] HybridClock initialized: now 1790284863858734 us; error 34 us; skew 500 ppm
Sep 24 21:21:03 dist-test-slave-cxpd krb5kdc[10085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284863, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:03.864902 10098 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:21:03.865373 10098 webserver.cc:460] Webserver started at http://127.20.115.254:38357/ using document root <none> and password file <none>
I20260924 21:21:03.865610 10098 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:03.865672 10098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:03.865813 10098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:03.866874 10098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "0b31583f6ab447f6acc56c7cca293f15"
format_stamp: "Formatted at 2026-09-24 21:21:03 on dist-test-slave-cxpd"
server_key: "648c83bd17552147835ab0d6d6299ff0"
server_key_iv: "241d89b14298036fb5dd52d16a877e30"
server_key_version: "encryptionkey@0"
I20260924 21:21:03.867238 10098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "0b31583f6ab447f6acc56c7cca293f15"
format_stamp: "Formatted at 2026-09-24 21:21:03 on dist-test-slave-cxpd"
server_key: "648c83bd17552147835ab0d6d6299ff0"
server_key_iv: "241d89b14298036fb5dd52d16a877e30"
server_key_version: "encryptionkey@0"
I20260924 21:21:03.868535 10098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:21:03.869372 10113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:03.869554 10098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260924 21:21:03.869653 10098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "0b31583f6ab447f6acc56c7cca293f15"
format_stamp: "Formatted at 2026-09-24 21:21:03 on dist-test-slave-cxpd"
server_key: "648c83bd17552147835ab0d6d6299ff0"
server_key_iv: "241d89b14298036fb5dd52d16a877e30"
server_key_version: "encryptionkey@0"
I20260924 21:21:03.869745 10098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:03.881886 10098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:03.884052 10098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:03.884208 10098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:03.888175 10098 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:37825
I20260924 21:21:03.888207 10165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:37825 every 8 connection(s)
I20260924 21:21:03.888528 10098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:21:03.889408 10166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:03.891530 10166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15: Bootstrap starting.
I20260924 21:21:03.892043 10166 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:03.892299 10166 log.cc:824] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:03.892477 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 10098
I20260924 21:21:03.892611 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:21:03.892867 20943 external_mini_cluster.cc:1428] Setting key 4ea6a9973d7f0b6da9709afcfc03b5da
I20260924 21:21:03.893126 10166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15: No bootstrap required, opened a new log
I20260924 21:21:03.895982 10166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b31583f6ab447f6acc56c7cca293f15" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37825 } }
I20260924 21:21:03.896152 10166 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:21:03.896183 10166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b31583f6ab447f6acc56c7cca293f15, State: Initialized, Role: FOLLOWER
I20260924 21:21:03.896296 10166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b31583f6ab447f6acc56c7cca293f15" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37825 } }
I20260924 21:21:03.896366 10166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:21:03.896401 10166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:21:03.896456 10166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:03.897040 10166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b31583f6ab447f6acc56c7cca293f15" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37825 } }
I20260924 21:21:03.897156 10166 leader_election.cc:302] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b31583f6ab447f6acc56c7cca293f15; no voters:
I20260924 21:21:03.897315 10166 leader_election.cc:288] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:21:03.897336 10171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:21:03.897446 10171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [term 1 LEADER]: Becoming Leader. State: Replica: 0b31583f6ab447f6acc56c7cca293f15, State: Running, Role: LEADER
I20260924 21:21:03.897526 10171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b31583f6ab447f6acc56c7cca293f15" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37825 } }
I20260924 21:21:03.897563 10166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:03.897930 10172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0b31583f6ab447f6acc56c7cca293f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b31583f6ab447f6acc56c7cca293f15" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37825 } } }
I20260924 21:21:03.898015 10172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:03.898397 10177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:21:03.898940 10173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0b31583f6ab447f6acc56c7cca293f15. Latest consensus state: current_term: 1 leader_uuid: "0b31583f6ab447f6acc56c7cca293f15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b31583f6ab447f6acc56c7cca293f15" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 37825 } } }
I20260924 21:21:03.899034 10173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15 [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:03.899574 10177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:21:03.900437 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:03.893297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60345 (local address 127.20.115.254:37825)
0924 21:21:03.893444 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:03.893454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:03.893487 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:21:03.894489 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:03.894498 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:03.894560 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:03.894726 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:03.894894 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:03.896351 (+ 1457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:03.897240 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:03.897865 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:03.898312 (+ 447us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:03.898746 (+ 434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:03.898750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:03.898750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:03.899082 (+ 332us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:03.899122 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:03.899138 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:03.899483 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:03.899529 (+ 46us) server_negotiation.cc:299] Negotiation successful
0924 21:21:03.899553 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":74,"threads_started":1}
I20260924 21:21:03.901396 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:21:03.905184 10177 catalog_manager.cc:1347] Generated new cluster ID: ad5d399c985a440c9472a999ba50650c
I20260924 21:21:03.905238 10177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:21:03.909776 10170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:03.902896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38003 (local address 127.20.115.254:37825)
0924 21:21:03.902933 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:03.902935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:03.903005 (+ 70us) server_negotiation.cc:407] Connection header received
0924 21:21:03.903072 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:03.903072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:03.903081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:03.903152 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:03.903235 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:03.903940 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:03.904678 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:03.905030 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:03.905099 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:03.909534 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:03.909535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:03.909536 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:03.909540 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:03.909554 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:03.909558 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:03.909613 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:03.909672 (+ 59us) server_negotiation.cc:299] Negotiation successful
0924 21:21:03.909683 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:21:03.926275 10177 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:21:03.926798 10177 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:21:03.933285 10177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0b31583f6ab447f6acc56c7cca293f15: Generated new TSK 0
I20260924 21:21:03.933493 10177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:21:03.941447 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 10098
W20260924 21:21:03.945817 10083 connection.cc:535] client connection to 127.20.115.254:37825 recv error: Network error: failed to read from TLS socket (remote: 127.20.115.254:37825): Connection reset by peer (error 104)
2026-09-24T21:21:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1324 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:21:03 dist-test-slave-cxpd krb5kdc[10197](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:21:03 dist-test-slave-cxpd krb5kdc[10197](info): set up 2 sockets
Sep 24 21:21:03 dist-test-slave-cxpd krb5kdc[10197](info): commencing operation
krb5kdc: starting...
W20260924 21:21:04.984899 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:21:05 dist-test-slave-cxpd krb5kdc[10197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:21:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:21:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:21:05.112709 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:34627
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:46271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:34627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:21:05.195094 10210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:05.195313 10210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:05.195335 10210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:05.196511 10210 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:05.196554 10210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:05.196569 10210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:05.196583 10210 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:05.196594 10210 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:05.198069 10210 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:46271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:34627
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:34627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:05.198323 10210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:05.198539 10210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:05.203284 10218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:05.203372 10215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:05.203559 10210 server_base.cc:1034] running on GCE node
W20260924 21:21:05.203385 10216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:05.203910 10210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:05.204205 10210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:05.205363 10210 hybrid_clock.cc:648] HybridClock initialized: now 1790284865205351 us; error 30 us; skew 500 ppm
Sep 24 21:21:05 dist-test-slave-cxpd krb5kdc[10197](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284865, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:05.207724 10210 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:21:05.208163 10210 webserver.cc:460] Webserver started at http://127.20.115.254:43383/ using document root <none> and password file <none>
I20260924 21:21:05.208381 10210 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:05.208446 10210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:05.208582 10210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:05.209671 10210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "fb9b70c8ecd44ec19927d7aa544aeebf"
format_stamp: "Formatted at 2026-09-24 21:21:05 on dist-test-slave-cxpd"
server_key: "f79ea3862675ea4a976b895a350f266f"
server_key_iv: "ac21b25068c1496425cdb180ab78f45e"
server_key_version: "encryptionkey@0"
I20260924 21:21:05.210024 10210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "fb9b70c8ecd44ec19927d7aa544aeebf"
format_stamp: "Formatted at 2026-09-24 21:21:05 on dist-test-slave-cxpd"
server_key: "f79ea3862675ea4a976b895a350f266f"
server_key_iv: "ac21b25068c1496425cdb180ab78f45e"
server_key_version: "encryptionkey@0"
I20260924 21:21:05.211309 10210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:21:05.212005 10225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:05.212167 10210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:21:05.212275 10210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "fb9b70c8ecd44ec19927d7aa544aeebf"
format_stamp: "Formatted at 2026-09-24 21:21:05 on dist-test-slave-cxpd"
server_key: "f79ea3862675ea4a976b895a350f266f"
server_key_iv: "ac21b25068c1496425cdb180ab78f45e"
server_key_version: "encryptionkey@0"
I20260924 21:21:05.212360 10210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:05.215592 10210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:05.217571 10210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:05.217722 10210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:05.221637 10210 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:34627
I20260924 21:21:05.221679 10277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:34627 every 8 connection(s)
I20260924 21:21:05.221987 10210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:21:05.222892 10278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:05.225114 10278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf: Bootstrap starting.
I20260924 21:21:05.225657 10278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:05.225934 10278 log.cc:824] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:05.226579 10278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf: No bootstrap required, opened a new log
I20260924 21:21:05.227624 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 10210
I20260924 21:21:05.227748 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:21:05.227937 20943 external_mini_cluster.cc:1428] Setting key ddb489ac0c5fc060bd41a3701f250c45
I20260924 21:21:05.228281 10278 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9b70c8ecd44ec19927d7aa544aeebf" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34627 } }
I20260924 21:21:05.228457 10278 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:21:05.228545 10278 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb9b70c8ecd44ec19927d7aa544aeebf, State: Initialized, Role: FOLLOWER
I20260924 21:21:05.228703 10278 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9b70c8ecd44ec19927d7aa544aeebf" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34627 } }
I20260924 21:21:05.228770 10278 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:21:05.228839 10278 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:21:05.228880 10278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:05.229871 10278 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9b70c8ecd44ec19927d7aa544aeebf" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34627 } }
I20260924 21:21:05.229995 10278 leader_election.cc:302] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb9b70c8ecd44ec19927d7aa544aeebf; no voters:
I20260924 21:21:05.230206 10278 leader_election.cc:288] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:21:05.230219 10283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:21:05.230276 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:05.228450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35513 (local address 127.20.115.254:34627)
0924 21:21:05.228628 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:05.228637 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:05.228668 (+ 31us) server_negotiation.cc:407] Connection header received
0924 21:21:05.228840 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:05.228848 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:05.228880 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:05.229012 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:05.229736 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:05.229740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:05.229740 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:05.229744 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:05.229788 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:05.229799 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:05.229873 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:05.229944 (+ 71us) server_negotiation.cc:299] Negotiation successful
0924 21:21:05.230072 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260924 21:21:05.230388 10283 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [term 1 LEADER]: Becoming Leader. State: Replica: fb9b70c8ecd44ec19927d7aa544aeebf, State: Running, Role: LEADER
I20260924 21:21:05.230508 10278 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:05.230484 10283 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9b70c8ecd44ec19927d7aa544aeebf" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34627 } }
I20260924 21:21:05.230903 10285 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb9b70c8ecd44ec19927d7aa544aeebf. Latest consensus state: current_term: 1 leader_uuid: "fb9b70c8ecd44ec19927d7aa544aeebf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9b70c8ecd44ec19927d7aa544aeebf" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34627 } } }
I20260924 21:21:05.230976 10284 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb9b70c8ecd44ec19927d7aa544aeebf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb9b70c8ecd44ec19927d7aa544aeebf" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 34627 } } }
I20260924 21:21:05.231055 10285 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:05.231132 10284 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:05.231204 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:21:05.231458 10292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:21:05.232049 10292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:21:05.233641 10292 catalog_manager.cc:1347] Generated new cluster ID: 43540c68a6da4a2fb796d5c9932c7b20
I20260924 21:21:05.233695 10292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:21:05.237247 10282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:05.232746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46809 (local address 127.20.115.254:34627)
0924 21:21:05.232780 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:05.232782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:05.233023 (+ 241us) server_negotiation.cc:407] Connection header received
0924 21:21:05.233083 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:05.233084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:05.233100 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:05.233157 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:05.237012 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:05.237014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:05.237014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:05.237019 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:05.237036 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:05.237041 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:05.237111 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:05.237168 (+ 57us) server_negotiation.cc:299] Negotiation successful
0924 21:21:05.237182 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:21:05.257071 10292 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:21:05.257258 10292 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:21:05.269968 10292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb9b70c8ecd44ec19927d7aa544aeebf: Generated new TSK 0
I20260924 21:21:05.270170 10292 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:21:05.274893 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 10210
2026-09-24T21:21:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1333 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:21:05 dist-test-slave-cxpd krb5kdc[10309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:21:05 dist-test-slave-cxpd krb5kdc[10309](info): set up 2 sockets
Sep 24 21:21:05 dist-test-slave-cxpd krb5kdc[10309](info): commencing operation
krb5kdc: starting...
W20260924 21:21:06.304924 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:21:06 dist-test-slave-cxpd krb5kdc[10309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:21:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:21:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:21:06.424546 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:38089
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:36989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:38089
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:21:06.504172 10322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:06.504372 10322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:06.504395 10322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:06.505548 10322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:06.505589 10322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:06.505604 10322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:06.505652 10322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:06.505673 10322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:06.507092 10322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:36989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:38089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:38089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:06.507330 10322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:06.507542 10322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:06.512231 10330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:06.512259 10328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:06.512282 10322 server_base.cc:1034] running on GCE node
W20260924 21:21:06.512286 10327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:06.512701 10322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:06.513015 10322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:06.514205 10322 hybrid_clock.cc:648] HybridClock initialized: now 1790284866514181 us; error 34 us; skew 500 ppm
Sep 24 21:21:06 dist-test-slave-cxpd krb5kdc[10309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284866, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:06.516601 10322 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:21:06.517019 10322 webserver.cc:460] Webserver started at http://127.20.115.254:37473/ using document root <none> and password file <none>
I20260924 21:21:06.517238 10322 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:06.517279 10322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:06.517364 10322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:06.518338 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "54a58cf42fbc43289d0cab2143488e45"
format_stamp: "Formatted at 2026-09-24 21:21:06 on dist-test-slave-cxpd"
server_key: "31b4985080bacd31f975c26e4fa8f7e4"
server_key_iv: "00aa6d23dd28291b0e83310ffa76a6ed"
server_key_version: "encryptionkey@0"
I20260924 21:21:06.518632 10322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "54a58cf42fbc43289d0cab2143488e45"
format_stamp: "Formatted at 2026-09-24 21:21:06 on dist-test-slave-cxpd"
server_key: "31b4985080bacd31f975c26e4fa8f7e4"
server_key_iv: "00aa6d23dd28291b0e83310ffa76a6ed"
server_key_version: "encryptionkey@0"
I20260924 21:21:06.519938 10322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:21:06.520649 10337 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:06.520830 10322 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260924 21:21:06.520917 10322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "54a58cf42fbc43289d0cab2143488e45"
format_stamp: "Formatted at 2026-09-24 21:21:06 on dist-test-slave-cxpd"
server_key: "31b4985080bacd31f975c26e4fa8f7e4"
server_key_iv: "00aa6d23dd28291b0e83310ffa76a6ed"
server_key_version: "encryptionkey@0"
I20260924 21:21:06.520974 10322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:06.540431 10322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:06.542407 10322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:06.542518 10322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:06.546337 10322 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:38089
I20260924 21:21:06.546411 10389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:38089 every 8 connection(s)
I20260924 21:21:06.546788 10322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:21:06.547649 10390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:06.549332 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 10322
I20260924 21:21:06.549460 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:21:06.549656 20943 external_mini_cluster.cc:1428] Setting key 1b9eb27aaa90e71bd35fe8446582ddce
I20260924 21:21:06.550702 10390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45: Bootstrap starting.
I20260924 21:21:06.551273 10390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:06.551549 10390 log.cc:824] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:06.552205 10390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45: No bootstrap required, opened a new log
I20260924 21:21:06.554001 10390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a58cf42fbc43289d0cab2143488e45" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38089 } }
I20260924 21:21:06.554164 10390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:21:06.554226 10390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54a58cf42fbc43289d0cab2143488e45, State: Initialized, Role: FOLLOWER
I20260924 21:21:06.554375 10390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a58cf42fbc43289d0cab2143488e45" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38089 } }
I20260924 21:21:06.554461 10390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:21:06.554497 10390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:21:06.554548 10390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:06.555083 10390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a58cf42fbc43289d0cab2143488e45" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38089 } }
I20260924 21:21:06.555174 10390 leader_election.cc:302] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54a58cf42fbc43289d0cab2143488e45; no voters:
I20260924 21:21:06.555380 10390 leader_election.cc:288] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:21:06.555425 10395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:21:06.555600 10395 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [term 1 LEADER]: Becoming Leader. State: Replica: 54a58cf42fbc43289d0cab2143488e45, State: Running, Role: LEADER
I20260924 21:21:06.555665 10390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:06.555729 10395 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a58cf42fbc43289d0cab2143488e45" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38089 } }
I20260924 21:21:06.556070 10395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54a58cf42fbc43289d0cab2143488e45. Latest consensus state: current_term: 1 leader_uuid: "54a58cf42fbc43289d0cab2143488e45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a58cf42fbc43289d0cab2143488e45" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38089 } } }
I20260924 21:21:06.556175 10395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:06.556213 10397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54a58cf42fbc43289d0cab2143488e45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54a58cf42fbc43289d0cab2143488e45" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 38089 } } }
Sep 24 21:21:06 dist-test-slave-cxpd krb5kdc[10309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:21:06.556300 10397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45 [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:06.556965 10402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:21:06.557468 10402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:21:06.559703 10402 catalog_manager.cc:1347] Generated new cluster ID: e748e1bf0f70443e848a290c754a5f99
I20260924 21:21:06.559753 10402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:21:06.560406 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:06.550175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42243 (local address 127.20.115.254:38089)
0924 21:21:06.550367 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:06.550376 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:06.550401 (+ 25us) server_negotiation.cc:407] Connection header received
0924 21:21:06.550855 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:06.550862 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:06.550912 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:06.551060 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:06.551781 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:06.552736 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:06.553523 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:06.553931 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:06.554055 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:06.556896 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:06.556900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:06.556900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:06.556913 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:06.558705 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:06.559119 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:06.559122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:06.559123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:06.559230 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:06.559442 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:06.559443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:06.559443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:06.559754 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:06.559905 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:06.560048 (+ 143us) server_negotiation.cc:299] Negotiation successful
0924 21:21:06.560190 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":83,"threads_started":1}
I20260924 21:21:06.561400 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:21:06.569931 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:06.562734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32997 (local address 127.20.115.254:38089)
0924 21:21:06.562772 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:06.562773 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:06.562950 (+ 177us) server_negotiation.cc:407] Connection header received
0924 21:21:06.562979 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:06.562980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:06.562991 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:06.563059 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:06.563619 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:06.564463 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:06.565467 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:06.565794 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:06.565872 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:06.567187 (+ 1315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:06.567189 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:06.567189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:06.567197 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:06.568810 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:06.569166 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:06.569167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:06.569168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:06.569230 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:06.569421 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:06.569421 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:06.569422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:06.569549 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:06.569645 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:06.569752 (+ 107us) server_negotiation.cc:299] Negotiation successful
0924 21:21:06.569839 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260924 21:21:06.587620 10402 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:21:06.588450 10402 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:21:06.591235 10402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54a58cf42fbc43289d0cab2143488e45: Generated new TSK 0
I20260924 21:21:06.591408 10402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:21:06.600878 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 10322
2026-09-24T21:21:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1325 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:21:06 dist-test-slave-cxpd krb5kdc[10421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:21:06 dist-test-slave-cxpd krb5kdc[10421](info): set up 2 sockets
Sep 24 21:21:06 dist-test-slave-cxpd krb5kdc[10421](info): commencing operation
krb5kdc: starting...
W20260924 21:21:07.637004 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:21:07 dist-test-slave-cxpd krb5kdc[10421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:21:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:21:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:21:07.759824 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33973
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:42207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:33973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:21:07.840989 10434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:07.841212 10434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:07.841236 10434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:07.842365 10434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:07.842401 10434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:07.842415 10434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:07.842427 10434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:07.842439 10434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:07.844063 10434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:42207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:33973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:33973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:07.844321 10434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:07.844527 10434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:07.849462 10442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:07.849479 10440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:07.849462 10439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:07.849927 10434 server_base.cc:1034] running on GCE node
I20260924 21:21:07.850204 10434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:07.850490 10434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:07.851655 10434 hybrid_clock.cc:648] HybridClock initialized: now 1790284867851631 us; error 37 us; skew 500 ppm
Sep 24 21:21:07 dist-test-slave-cxpd krb5kdc[10421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284867, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:07.854142 10434 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:21:07.854586 10434 webserver.cc:460] Webserver started at http://127.20.115.254:46739/ using document root <none> and password file <none>
I20260924 21:21:07.854817 10434 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:07.854862 10434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:07.855012 10434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:07.856065 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "91d5a742fec54ea5ade36b8debaff2ff"
format_stamp: "Formatted at 2026-09-24 21:21:07 on dist-test-slave-cxpd"
server_key: "1b1273d82b790f8d594b9011ef6ea0a7"
server_key_iv: "3f878e652dec00ef3bfdee5461d26d34"
server_key_version: "encryptionkey@0"
I20260924 21:21:07.856415 10434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "91d5a742fec54ea5ade36b8debaff2ff"
format_stamp: "Formatted at 2026-09-24 21:21:07 on dist-test-slave-cxpd"
server_key: "1b1273d82b790f8d594b9011ef6ea0a7"
server_key_iv: "3f878e652dec00ef3bfdee5461d26d34"
server_key_version: "encryptionkey@0"
I20260924 21:21:07.857810 10434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:21:07.858798 10449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:07.859023 10434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:21:07.859127 10434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "91d5a742fec54ea5ade36b8debaff2ff"
format_stamp: "Formatted at 2026-09-24 21:21:07 on dist-test-slave-cxpd"
server_key: "1b1273d82b790f8d594b9011ef6ea0a7"
server_key_iv: "3f878e652dec00ef3bfdee5461d26d34"
server_key_version: "encryptionkey@0"
I20260924 21:21:07.859222 10434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:07.872220 10434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:07.874495 10434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:07.874711 10434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:07.878708 10434 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:33973
I20260924 21:21:07.878973 10501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:33973 every 8 connection(s)
I20260924 21:21:07.879235 10434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:21:07.880008 10502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:07.882375 10502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff: Bootstrap starting.
I20260924 21:21:07.882956 10502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:07.883246 10502 log.cc:824] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:07.883929 10502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff: No bootstrap required, opened a new log
I20260924 21:21:07.884891 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 10434
I20260924 21:21:07.885022 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:21:07.885229 20943 external_mini_cluster.cc:1428] Setting key 313859f2015325a77361ba3bc5448a8d
I20260924 21:21:07.886054 10502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d5a742fec54ea5ade36b8debaff2ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33973 } }
I20260924 21:21:07.886260 10502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:21:07.886296 10502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91d5a742fec54ea5ade36b8debaff2ff, State: Initialized, Role: FOLLOWER
I20260924 21:21:07.886420 10502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d5a742fec54ea5ade36b8debaff2ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33973 } }
I20260924 21:21:07.886509 10502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:21:07.886541 10502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:21:07.886636 10502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:07.887200 10502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d5a742fec54ea5ade36b8debaff2ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33973 } }
I20260924 21:21:07.887328 10502 leader_election.cc:302] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91d5a742fec54ea5ade36b8debaff2ff; no voters:
I20260924 21:21:07.887487 10502 leader_election.cc:288] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:21:07.887516 10507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:21:07.887653 10507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [term 1 LEADER]: Becoming Leader. State: Replica: 91d5a742fec54ea5ade36b8debaff2ff, State: Running, Role: LEADER
I20260924 21:21:07.887786 10502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:07.887797 10507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d5a742fec54ea5ade36b8debaff2ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33973 } }
I20260924 21:21:07.888304 10509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91d5a742fec54ea5ade36b8debaff2ff. Latest consensus state: current_term: 1 leader_uuid: "91d5a742fec54ea5ade36b8debaff2ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d5a742fec54ea5ade36b8debaff2ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33973 } } }
I20260924 21:21:07.888408 10509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:07.888612 10508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91d5a742fec54ea5ade36b8debaff2ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91d5a742fec54ea5ade36b8debaff2ff" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 33973 } } }
I20260924 21:21:07.888921 10508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:07.889106 10516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:21:07.889626 10516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:21:07.891300 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:07.885761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60163 (local address 127.20.115.254:33973)
0924 21:21:07.886153 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:07.886162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:07.886191 (+ 29us) server_negotiation.cc:407] Connection header received
0924 21:21:07.886317 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:07.886324 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:07.886378 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:07.886529 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:07.886791 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:07.887871 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:07.888993 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:07.889611 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:07.889737 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:07.890447 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:07.890451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:07.890451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:07.890664 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:07.890703 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:07.890718 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:07.890778 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:07.890893 (+ 115us) server_negotiation.cc:299] Negotiation successful
0924 21:21:07.891065 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260924 21:21:07.891722 10516 catalog_manager.cc:1347] Generated new cluster ID: 0780e9ef69cb4d748579895bfca300e1
I20260924 21:21:07.891820 10516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:21:07.892134 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:21:07.896248 10506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:07.893526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44891 (local address 127.20.115.254:33973)
0924 21:21:07.893603 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:07.893605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:07.893653 (+ 48us) server_negotiation.cc:407] Connection header received
0924 21:21:07.893723 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:07.893724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:07.893734 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:07.893787 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:07.893915 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:07.894361 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:07.895098 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:07.895300 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:07.895380 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:07.896016 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:07.896017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:07.896018 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:07.896022 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:07.896034 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:07.896038 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:07.896085 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:07.896149 (+ 64us) server_negotiation.cc:299] Negotiation successful
0924 21:21:07.896172 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260924 21:21:07.905483 10516 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:21:07.906241 10516 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:21:07.910072 10516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91d5a742fec54ea5ade36b8debaff2ff: Generated new TSK 0
I20260924 21:21:07.910256 10516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:21:07.918452 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 10434
2026-09-24T21:21:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1317 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:21:07 dist-test-slave-cxpd krb5kdc[10533](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:21:07 dist-test-slave-cxpd krb5kdc[10533](info): set up 2 sockets
Sep 24 21:21:07 dist-test-slave-cxpd krb5kdc[10533](info): commencing operation
krb5kdc: starting...
W20260924 21:21:08.957056 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:21:09 dist-test-slave-cxpd krb5kdc[10533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:21:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:21:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:21:09.075822 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:32915
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:43301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:32915
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:21:09.156843 10546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:09.157052 10546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:09.157075 10546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:09.158255 10546 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:09.158298 10546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:09.158313 10546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:09.158326 10546 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:09.158339 10546 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:09.159835 10546 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:43301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:32915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:32915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:09.160081 10546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:09.160320 10546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:09.165324 10552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:09.165555 10546 server_base.cc:1034] running on GCE node
W20260924 21:21:09.165515 10554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:09.165495 10551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:09.166065 10546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:09.166319 10546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:09.167498 10546 hybrid_clock.cc:648] HybridClock initialized: now 1790284869167482 us; error 32 us; skew 500 ppm
Sep 24 21:21:09 dist-test-slave-cxpd krb5kdc[10533](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284869, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:09.177129 10546 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:21:09.177645 10546 webserver.cc:460] Webserver started at http://127.20.115.254:34205/ using document root <none> and password file <none>
I20260924 21:21:09.177896 10546 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:09.177944 10546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:09.178117 10546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:09.179242 10546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "09f691da94ab4f8ba67c41b7eb0f6f36"
format_stamp: "Formatted at 2026-09-24 21:21:09 on dist-test-slave-cxpd"
server_key: "d055c954c522570edef6240c82dad56e"
server_key_iv: "5f625da38e7e1e9730d850d16c25c136"
server_key_version: "encryptionkey@0"
I20260924 21:21:09.179615 10546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "09f691da94ab4f8ba67c41b7eb0f6f36"
format_stamp: "Formatted at 2026-09-24 21:21:09 on dist-test-slave-cxpd"
server_key: "d055c954c522570edef6240c82dad56e"
server_key_iv: "5f625da38e7e1e9730d850d16c25c136"
server_key_version: "encryptionkey@0"
I20260924 21:21:09.181169 10546 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:21:09.182194 10561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:09.182447 10546 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260924 21:21:09.182564 10546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "09f691da94ab4f8ba67c41b7eb0f6f36"
format_stamp: "Formatted at 2026-09-24 21:21:09 on dist-test-slave-cxpd"
server_key: "d055c954c522570edef6240c82dad56e"
server_key_iv: "5f625da38e7e1e9730d850d16c25c136"
server_key_version: "encryptionkey@0"
I20260924 21:21:09.182647 10546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:09.186189 10546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:09.188364 10546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:09.188553 10546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:09.192903 10546 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:32915
I20260924 21:21:09.192939 10613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:32915 every 8 connection(s)
I20260924 21:21:09.193284 10546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:21:09.194334 10614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:09.196558 10614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36: Bootstrap starting.
I20260924 21:21:09.197204 10614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:09.197485 10614 log.cc:824] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:09.198197 10614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36: No bootstrap required, opened a new log
I20260924 21:21:09.199723 10614 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f691da94ab4f8ba67c41b7eb0f6f36" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 32915 } }
I20260924 21:21:09.199859 10614 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260924 21:21:09.199882 10614 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09f691da94ab4f8ba67c41b7eb0f6f36, State: Initialized, Role: FOLLOWER
I20260924 21:21:09.200042 10614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f691da94ab4f8ba67c41b7eb0f6f36" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 32915 } }
I20260924 21:21:09.200104 10614 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260924 21:21:09.200125 10614 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260924 21:21:09.200181 10614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:09.200979 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 10546
I20260924 21:21:09.201074 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:21:09.201126 10614 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f691da94ab4f8ba67c41b7eb0f6f36" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 32915 } }
I20260924 21:21:09.201272 10614 leader_election.cc:302] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09f691da94ab4f8ba67c41b7eb0f6f36; no voters:
I20260924 21:21:09.201306 20943 external_mini_cluster.cc:1428] Setting key fa7fe37eef087d24f4dc0e26a8f0ff44
I20260924 21:21:09.201452 10614 leader_election.cc:288] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260924 21:21:09.201478 10617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:21:09.201690 10617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [term 1 LEADER]: Becoming Leader. State: Replica: 09f691da94ab4f8ba67c41b7eb0f6f36, State: Running, Role: LEADER
I20260924 21:21:09.201793 10614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:09.201817 10617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f691da94ab4f8ba67c41b7eb0f6f36" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 32915 } }
I20260924 21:21:09.202337 10618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09f691da94ab4f8ba67c41b7eb0f6f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f691da94ab4f8ba67c41b7eb0f6f36" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 32915 } } }
I20260924 21:21:09.202456 10618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:09.202333 10619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09f691da94ab4f8ba67c41b7eb0f6f36. Latest consensus state: current_term: 1 leader_uuid: "09f691da94ab4f8ba67c41b7eb0f6f36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09f691da94ab4f8ba67c41b7eb0f6f36" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 32915 } } }
I20260924 21:21:09.202593 10619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36 [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:09.202745 10628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:21:09.203343 10628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:21:09.204208 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:09.202012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47855 (local address 127.20.115.254:32915)
0924 21:21:09.202256 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:09.202265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:09.202298 (+ 33us) server_negotiation.cc:407] Connection header received
0924 21:21:09.202730 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:09.202739 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:09.202772 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:09.203210 (+ 438us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:09.203703 (+ 493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:09.203707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:09.203707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:09.203711 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:09.203749 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:09.203764 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:09.203855 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:09.203961 (+ 106us) server_negotiation.cc:299] Negotiation successful
0924 21:21:09.204004 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
I20260924 21:21:09.205120 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:21:09.205279 10628 catalog_manager.cc:1347] Generated new cluster ID: a5c9e280b20c4ccaa7b0223769fcd689
I20260924 21:21:09.205320 10628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:21:09.213835 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:09.206620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46997 (local address 127.20.115.254:32915)
0924 21:21:09.212869 (+ 6249us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:09.212871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:09.212881 (+ 10us) server_negotiation.cc:407] Connection header received
0924 21:21:09.212904 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:09.212905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:09.212909 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:09.212974 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:09.213604 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:09.213605 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:09.213605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0924 21:21:09.213609 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:09.213626 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0924 21:21:09.213631 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:09.213699 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:09.213746 (+ 47us) server_negotiation.cc:299] Negotiation successful
0924 21:21:09.213759 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6225}
I20260924 21:21:09.217217 10628 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:21:09.217367 10628 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:21:09.224704 10628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09f691da94ab4f8ba67c41b7eb0f6f36: Generated new TSK 0
I20260924 21:21:09.224920 10628 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:21:09.227435 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 10546
2026-09-24T21:21:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1308 ms)
[----------] 18 tests from AuthTokenIssuingTest (23801 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:21:09 dist-test-slave-cxpd krb5kdc[10645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:21:09 dist-test-slave-cxpd krb5kdc[10645](info): set up 2 sockets
Sep 24 21:21:09 dist-test-slave-cxpd krb5kdc[10645](info): commencing operation
krb5kdc: starting...
W20260924 21:21:10.260963 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:21:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:21:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:21:10.379632 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:44609
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:44609,127.20.115.253:35343,127.20.115.252:36397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:21:10.460377 10658 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:21:10.460589 10658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:10.460608 10658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:10.460626 10658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:10.461866 10658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:10.461913 10658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:10.461928 10658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:10.461941 10658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:10.461952 10658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:10.463464 10658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:44609,127.20.115.253:35343,127.20.115.252:36397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:44609
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:10.463716 10658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:10.463934 10658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:10.468755 10666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:10.468809 10663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:10.468755 10664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:10.468827 10658 server_base.cc:1034] running on GCE node
I20260924 21:21:10.469168 10658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:10.469451 10658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:10.470593 10658 hybrid_clock.cc:648] HybridClock initialized: now 1790284870470582 us; error 28 us; skew 500 ppm
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:10.473026 10658 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:21:10.473449 10658 webserver.cc:460] Webserver started at http://127.20.115.254:41901/ using document root <none> and password file <none>
I20260924 21:21:10.473687 10658 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:10.473762 10658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:10.473910 10658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:10.474957 10658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "f8aaa7536d824df5be091f2e07b67e1c"
format_stamp: "Formatted at 2026-09-24 21:21:10 on dist-test-slave-cxpd"
server_key: "812eb8512aa8a0febecfcf93354136e2"
server_key_iv: "7ec25eaba53ea1de32685beb13613091"
server_key_version: "encryptionkey@0"
I20260924 21:21:10.475339 10658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "f8aaa7536d824df5be091f2e07b67e1c"
format_stamp: "Formatted at 2026-09-24 21:21:10 on dist-test-slave-cxpd"
server_key: "812eb8512aa8a0febecfcf93354136e2"
server_key_iv: "7ec25eaba53ea1de32685beb13613091"
server_key_version: "encryptionkey@0"
I20260924 21:21:10.476642 10658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260924 21:21:10.477466 10673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:10.477672 10658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:21:10.477778 10658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "f8aaa7536d824df5be091f2e07b67e1c"
format_stamp: "Formatted at 2026-09-24 21:21:10 on dist-test-slave-cxpd"
server_key: "812eb8512aa8a0febecfcf93354136e2"
server_key_iv: "7ec25eaba53ea1de32685beb13613091"
server_key_version: "encryptionkey@0"
I20260924 21:21:10.477864 10658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:10.485633 10658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:10.487294 10658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:10.487452 10658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:10.491173 10658 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:44609
I20260924 21:21:10.491205 10725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:44609 every 8 connection(s)
I20260924 21:21:10.491524 10658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:21:10.492369 10726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:10.494004 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 10658
I20260924 21:21:10.494112 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:21:10.493973 10726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:10.494307 20943 external_mini_cluster.cc:1428] Setting key ab04927b00828ad494e5e5b91f6b1cc8
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.115.253@KRBTEST.COM; defaulting to no policy
I20260924 21:21:10.504812 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.494787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41682 (local address 127.20.115.254:44609)
0924 21:21:10.495561 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.495564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.495575 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:21:10.495632 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.495633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.495647 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.495748 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.496291 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.497164 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.498486 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.498869 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.498959 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.501313 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.501319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.501320 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.501333 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.503081 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.503514 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.503517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.503518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.503605 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.503850 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.503851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.503852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.504079 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.504178 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.504397 (+ 219us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.504546 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":751,"thread_start_us":73,"threads_started":1}
I20260924 21:21:10.504822 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.494760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:44609 (local address 127.0.0.1:41682)
0924 21:21:10.494923 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.494963 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:10.495417 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:10.495768 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:10.495770 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:10.496042 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:10.496185 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.496189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.497279 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.497280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.498393 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.498396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.498972 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.498973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.499059 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.499546 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:10.499558 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:10.501207 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:10.503188 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.503195 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.503197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.503409 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.503683 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.503684 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.503684 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.503777 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.504173 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:10.504178 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:10.504244 (+ 66us) client_negotiation.cc:769] Sending connection context
0924 21:21:10.504362 (+ 118us) client_negotiation.cc:240] Negotiation successful
0924 21:21:10.504615 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":109,"thread_start_us":81,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.253@KRBTEST.COM" created.
I20260924 21:21:10.505970 10726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:10.506341 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.506219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35343 (local address 127.0.0.1:42978)
0924 21:21:10.506270 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.506311 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260924 21:21:10.506471 10674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
I20260924 21:21:10.506878 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.506785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35343 (local address 127.0.0.1:42980)
0924 21:21:10.506833 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.506856 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 21:21:10.506995 10726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.253:35343: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab.
Entry for principal kudu/127.20.115.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab.
I20260924 21:21:10.519968 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.253:35343
--webserver_interface=127.20.115.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab
--principal=kudu/127.20.115.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:44609,127.20.115.253:35343,127.20.115.252:36397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:21:10.556244 10726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } attempt: 1
I20260924 21:21:10.556761 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.556620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35343 (local address 127.0.0.1:42994)
0924 21:21:10.556690 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.556731 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260924 21:21:10.557152 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.557037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35343 (local address 127.0.0.1:42996)
0924 21:21:10.557103 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.557135 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260924 21:21:10.557236 10726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.253:35343: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
W20260924 21:21:10.600843 10733 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:21:10.601065 10733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:10.601081 10733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:10.601102 10733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:10.602281 10733 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:10.602324 10733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:10.602339 10733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:10.602350 10733 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:10.602362 10733 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:10.603879 10733 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:44609,127.20.115.253:35343,127.20.115.252:36397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab
--principal=kudu/127.20.115.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.253:35343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.115.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:10.604130 10733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:10.604379 10733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:10.609182 10739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:10.609223 10738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:10.609270 10733 server_base.cc:1034] running on GCE node
W20260924 21:21:10.609196 10741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:10.609629 10733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:10.609944 10733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:10.611124 10733 hybrid_clock.cc:648] HybridClock initialized: now 1790284870611112 us; error 38 us; skew 500 ppm
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:10.613643 10733 init.cc:377] Logged in from keytab as kudu/127.20.115.253@KRBTEST.COM (short username kudu)
I20260924 21:21:10.614027 10733 webserver.cc:460] Webserver started at http://127.20.115.253:40469/ using document root <none> and password file <none>
I20260924 21:21:10.614269 10733 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:10.614336 10733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:10.614480 10733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:10.615556 10733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/data/instance:
uuid: "c1e2e22c345f40e0a9eb14e74fbcace6"
format_stamp: "Formatted at 2026-09-24 21:21:10 on dist-test-slave-cxpd"
server_key: "cfe2391842f6ef128edcac2037f5b01b"
server_key_iv: "f7a07d28e921d04eb2ba1d72c9d28f83"
server_key_version: "encryptionkey@0"
I20260924 21:21:10.615931 10733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/wal/instance:
uuid: "c1e2e22c345f40e0a9eb14e74fbcace6"
format_stamp: "Formatted at 2026-09-24 21:21:10 on dist-test-slave-cxpd"
server_key: "cfe2391842f6ef128edcac2037f5b01b"
server_key_iv: "f7a07d28e921d04eb2ba1d72c9d28f83"
server_key_version: "encryptionkey@0"
I20260924 21:21:10.617367 10733 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:21:10.618171 10748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:10.618374 10733 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:21:10.618481 10733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/wal
uuid: "c1e2e22c345f40e0a9eb14e74fbcace6"
format_stamp: "Formatted at 2026-09-24 21:21:10 on dist-test-slave-cxpd"
server_key: "cfe2391842f6ef128edcac2037f5b01b"
server_key_iv: "f7a07d28e921d04eb2ba1d72c9d28f83"
server_key_version: "encryptionkey@0"
I20260924 21:21:10.618568 10733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:10.625367 10726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } attempt: 2
I20260924 21:21:10.625777 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.625658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35343 (local address 127.0.0.1:43000)
0924 21:21:10.625713 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.625749 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260924 21:21:10.626138 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.626038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35343 (local address 127.0.0.1:43004)
0924 21:21:10.626085 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.626118 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 21:21:10.626230 10726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.253:35343: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35343: connect: Connection refused (error 111)
I20260924 21:21:10.630673 10733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:10.632433 10733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:10.632609 10733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:10.636533 10733 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.253:35343
I20260924 21:21:10.636561 10800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.253:35343 every 8 connection(s)
I20260924 21:21:10.636997 10733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
I20260924 21:21:10.637806 10801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:10.639222 10801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:21:10.644822 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 10733
I20260924 21:21:10.644943 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-1/wal/instance
I20260924 21:21:10.645133 20943 external_mini_cluster.cc:1428] Setting key e5c8133268dcc538a4f6860a1ddf9a31
I20260924 21:21:10.648114 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.639853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41698 (local address 127.20.115.254:44609)
0924 21:21:10.639909 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.639913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.640121 (+ 208us) server_negotiation.cc:407] Connection header received
0924 21:21:10.640577 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.640578 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.640590 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.640645 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.641289 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.641770 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.642668 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.642855 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.642909 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.645062 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.645064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.645065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.645073 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.646703 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.647196 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.647197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.647197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.647249 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.647536 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.647536 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.647536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.647638 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.647687 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.647991 (+ 304us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.648056 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:21:10.648409 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.639844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:44609 (local address 127.0.0.1:41698)
0924 21:21:10.639999 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.640039 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:10.640495 (+ 456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:10.640700 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:10.640703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:10.641016 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:10.641200 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.641204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.641848 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.641848 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.642600 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.642602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.642927 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.642929 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.643027 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.643533 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:10.643541 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:10.644936 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:10.646790 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.646796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.646799 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.647114 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.647323 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.647324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.647324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.647465 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.647701 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:10.647706 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:10.647862 (+ 156us) client_negotiation.cc:769] Sending connection context
0924 21:21:10.647975 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:21:10.648132 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260924 21:21:10.649036 10801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.20.115.252@KRBTEST.COM; defaulting to no policy
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.252@KRBTEST.COM" created.
I20260924 21:21:10.656239 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.649315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35343 (local address 127.0.0.1:43020)
0924 21:21:10.649374 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.649383 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:10.649426 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:10.649669 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:10.649670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:10.649811 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:10.649851 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.649853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.650717 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.650718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.651215 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.651216 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.651791 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.651792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.651859 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.652265 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:10.652270 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:10.653485 (+ 1215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:10.655128 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.655128 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.655129 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.655321 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.655558 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.655558 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.655559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.655633 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.655900 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:10.655900 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:10.655936 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 21:21:10.656024 (+ 88us) client_negotiation.cc:240] Negotiation successful
0924 21:21:10.656118 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260924 21:21:10.656307 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.649383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43020 (local address 127.20.115.253:35343)
0924 21:21:10.649545 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.649549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.649564 (+ 15us) server_negotiation.cc:407] Connection header received
0924 21:21:10.649579 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.649581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.649590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.649668 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.649933 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.650612 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.651314 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.651712 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.651793 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.653591 (+ 1798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.653598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.653598 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.653605 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.655033 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.655414 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.655416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.655417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.655470 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.655699 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.655700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.655701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.655830 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.655920 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.656056 (+ 136us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.656178 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260924 21:21:10.656818 10801 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:10.657196 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.657083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60686)
0924 21:21:10.657137 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.657179 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 21:21:10.657361 10751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
I20260924 21:21:10.657783 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.657687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60696)
0924 21:21:10.657735 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.657762 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260924 21:21:10.657905 10801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:36397: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab.
Entry for principal kudu/127.20.115.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab.
I20260924 21:21:10.671563 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.252:36397
--webserver_interface=127.20.115.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab
--principal=kudu/127.20.115.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:34845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:44609,127.20.115.253:35343,127.20.115.252:36397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:21:10.707124 10801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } attempt: 1
I20260924 21:21:10.707612 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.707480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60708)
0924 21:21:10.707545 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.707582 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 21:21:10.708025 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.707924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60724)
0924 21:21:10.707979 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.708007 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260924 21:21:10.708122 10801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:36397: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
I20260924 21:21:10.717388 10726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } attempt: 3
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
I20260924 21:21:10.724900 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.717772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35343 (local address 127.0.0.1:43032)
0924 21:21:10.717834 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.717843 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:10.717918 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:10.718113 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:10.718114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:10.718278 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:10.718347 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.718349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.719012 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.719013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.719554 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.719556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.719939 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.719941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.720020 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.720481 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:10.720486 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:10.722007 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:10.723883 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.723884 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.723884 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.724087 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.724297 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.724297 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.724298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.724352 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.724614 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:10.724614 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:10.724653 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 21:21:10.724706 (+ 53us) client_negotiation.cc:240] Negotiation successful
0924 21:21:10.724827 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260924 21:21:10.724942 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.717840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.20.115.253:35343)
0924 21:21:10.717900 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.717903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.717914 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:21:10.717987 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.717988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.718007 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.718104 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.718428 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.718927 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.719640 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.719852 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.719939 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.722128 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.722130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.722131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.722138 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.723793 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.724161 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.724162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.724162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.724226 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.724414 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.724415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.724416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.724550 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.724619 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.724747 (+ 128us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.724850 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:21:10.725405 10726 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:10.725739 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.725639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60726)
0924 21:21:10.725685 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.725714 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260924 21:21:10.726105 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.726012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60730)
0924 21:21:10.726063 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.726086 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260924 21:21:10.726220 10726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:36397: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
W20260924 21:21:10.754556 10808 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:21:10.754769 10808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:10.754788 10808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:10.754808 10808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:10.755995 10808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:10.756032 10808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:10.756047 10808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:10.756059 10808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:10.756071 10808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:10.757349 10726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } attempt: 1
I20260924 21:21:10.757475 10808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:34845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:44609,127.20.115.253:35343,127.20.115.252:36397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab
--principal=kudu/127.20.115.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.252:36397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.115.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:10.757771 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.757641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60740)
0924 21:21:10.757697 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.757732 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:21:10.757774 10808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:10.758062 10808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260924 21:21:10.758242 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.758099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60742)
0924 21:21:10.758183 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.758213 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260924 21:21:10.758358 10726 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:36397: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
W20260924 21:21:10.762732 10813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:10.762763 10814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:10.762753 10816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:10.762827 10808 server_base.cc:1034] running on GCE node
I20260924 21:21:10.763150 10808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:10.763453 10808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:10.764635 10808 hybrid_clock.cc:648] HybridClock initialized: now 1790284870764634 us; error 38 us; skew 500 ppm
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:10.767086 10808 init.cc:377] Logged in from keytab as kudu/127.20.115.252@KRBTEST.COM (short username kudu)
I20260924 21:21:10.767488 10808 webserver.cc:460] Webserver started at http://127.20.115.252:35945/ using document root <none> and password file <none>
I20260924 21:21:10.767726 10808 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:10.767792 10808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:10.767937 10808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:10.768988 10808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/data/instance:
uuid: "9ae39f663cc040ddae69eb339debad4d"
format_stamp: "Formatted at 2026-09-24 21:21:10 on dist-test-slave-cxpd"
server_key: "b56fdf8e407ccb97031035379f746b41"
server_key_iv: "d929c672759583353e561220532ed80c"
server_key_version: "encryptionkey@0"
I20260924 21:21:10.769327 10808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/wal/instance:
uuid: "9ae39f663cc040ddae69eb339debad4d"
format_stamp: "Formatted at 2026-09-24 21:21:10 on dist-test-slave-cxpd"
server_key: "b56fdf8e407ccb97031035379f746b41"
server_key_iv: "d929c672759583353e561220532ed80c"
server_key_version: "encryptionkey@0"
I20260924 21:21:10.770660 10808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:21:10.771359 10823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:10.771523 10808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:10.771622 10808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/wal
uuid: "9ae39f663cc040ddae69eb339debad4d"
format_stamp: "Formatted at 2026-09-24 21:21:10 on dist-test-slave-cxpd"
server_key: "b56fdf8e407ccb97031035379f746b41"
server_key_iv: "d929c672759583353e561220532ed80c"
server_key_version: "encryptionkey@0"
I20260924 21:21:10.771692 10808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:10.776299 10801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } attempt: 2
I20260924 21:21:10.776654 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.776537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60746)
0924 21:21:10.776594 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.776628 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:21:10.777047 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.776952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60754)
0924 21:21:10.776996 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.777020 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260924 21:21:10.777130 10801 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:36397: Network error: Client connection negotiation failed: client connection to 127.20.115.252:36397: connect: Connection refused (error 111)
I20260924 21:21:10.790055 10808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:10.792181 10808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:10.792335 10808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:10.796118 10808 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.252:36397
I20260924 21:21:10.796165 10875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.252:36397 every 8 connection(s)
I20260924 21:21:10.796516 10808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
I20260924 21:21:10.797362 10876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:10.798753 10876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:21:10.806282 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 10808
I20260924 21:21:10.806393 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790284606872664-20943-0/minicluster-data/master-2/wal/instance
I20260924 21:21:10.806577 20943 external_mini_cluster.cc:1428] Setting key 9f45f5a46a56e1bd293a1f1db55e416b
I20260924 21:21:10.806867 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.799382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41700 (local address 127.20.115.254:44609)
0924 21:21:10.799433 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.799435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.799649 (+ 214us) server_negotiation.cc:407] Connection header received
0924 21:21:10.800055 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.800056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.800065 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.800109 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.800664 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.801166 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.802071 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.802259 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.802320 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.804266 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.804267 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.804268 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.804273 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.805487 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.805898 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.805899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.805899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.805946 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.806179 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.806180 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.806181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.806321 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.806399 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.806702 (+ 303us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.806780 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:21:10.807030 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.799368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:44609 (local address 127.0.0.1:41700)
0924 21:21:10.799531 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.799566 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:10.799979 (+ 413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:10.800162 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:10.800164 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:10.800389 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:10.800562 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.800566 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.801246 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.801246 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.801985 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.801987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.802337 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.802337 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.802433 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.802910 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:10.802917 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:10.804191 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:10.805576 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.805582 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.805584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.805832 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.806003 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.806003 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.806003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.806095 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.806398 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:10.806403 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:10.806554 (+ 151us) client_negotiation.cc:769] Sending connection context
0924 21:21:10.806666 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 21:21:10.806807 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260924 21:21:10.807693 10876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
I20260924 21:21:10.814802 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.807996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.20.115.253:35343)
0924 21:21:10.808075 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.808078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.808089 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:21:10.808114 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.808115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.808127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.808183 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.808449 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.808965 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.809688 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.809990 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.810064 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.812036 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.812037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.812038 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.812046 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.813668 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.814127 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.814128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.814128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.814184 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.814381 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.814382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.814382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.814475 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.814521 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.814688 (+ 167us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.814743 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260924 21:21:10.814905 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.807913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35343 (local address 127.0.0.1:43034)
0924 21:21:10.807960 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.807973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:10.808011 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:10.808200 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:10.808201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:10.808302 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:10.808368 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.808369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.809032 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.809033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.809611 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.809612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.810070 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.810071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.810161 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.810489 (+ 328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:10.810493 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:10.811947 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:10.813759 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.813760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.813760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.814043 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.814250 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.814251 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.814251 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.814318 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.814535 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:10.814536 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:10.814579 (+ 43us) client_negotiation.cc:769] Sending connection context
0924 21:21:10.814672 (+ 93us) client_negotiation.cc:240] Negotiation successful
0924 21:21:10.814799 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 21:21:10.815243 10876 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:10.816437 10880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.807069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60760 (local address 127.20.115.252:36397)
0924 21:21:10.807215 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.807219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.807473 (+ 254us) server_negotiation.cc:407] Connection header received
0924 21:21:10.807502 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.807504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.807517 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.807603 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.808357 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.809236 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.810125 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.810565 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.810637 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.813009 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.813016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.813016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.813023 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.814900 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.815344 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.815347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.815348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.815414 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.815682 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.815683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.815683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.815860 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.815992 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.816241 (+ 249us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.816308 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:21:10.823184 10878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.815508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60774)
0924 21:21:10.815644 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.815652 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:10.815680 (+ 28us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:10.815814 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:10.815814 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:10.815915 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:10.815941 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.815942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.816811 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.816811 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.817343 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.817345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.817868 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.817869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.817904 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.818282 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:10.818286 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:10.819637 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:10.821967 (+ 2330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.821968 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.821969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.822129 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.822395 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.822395 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.822396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.822447 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.822829 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:10.822830 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:10.822875 (+ 45us) client_negotiation.cc:769] Sending connection context
0924 21:21:10.822976 (+ 101us) client_negotiation.cc:240] Negotiation successful
0924 21:21:10.823077 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260924 21:21:10.823273 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.815541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60774 (local address 127.20.115.252:36397)
0924 21:21:10.815705 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.815707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.815713 (+ 6us) server_negotiation.cc:407] Connection header received
0924 21:21:10.815730 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.815730 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.815740 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.815814 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.816006 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.816703 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.817421 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.817792 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.817870 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.819720 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.819721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.819722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.819729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.821880 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.822251 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.822252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.822252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.822305 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.822516 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.822517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.822517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.822654 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.822837 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.823000 (+ 163us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.823127 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260924 21:21:10.824745 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.817247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41712 (local address 127.20.115.254:44609)
0924 21:21:10.817288 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.817290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.817299 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:21:10.817321 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.817322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.817330 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.817404 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.817952 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.818805 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.819747 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.819926 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.820000 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.822012 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.822013 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.822013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.822020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.823682 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.824051 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.824052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.824052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.824106 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.824315 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.824316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.824316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.824409 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.824537 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.824622 (+ 85us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.824688 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260924 21:21:10.824800 10876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d: Bootstrap starting.
I20260924 21:21:10.825470 10876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:10.825757 10876 log.cc:824] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:10.826452 10876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d: No bootstrap required, opened a new log
I20260924 21:21:10.827786 10876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } }
I20260924 21:21:10.827924 10876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ae39f663cc040ddae69eb339debad4d, State: Initialized, Role: FOLLOWER
I20260924 21:21:10.828011 10876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } }
I20260924 21:21:10.828290 10883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.828351 10876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:10.828394 10883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [sys.catalog]: This master's current role is: FOLLOWER
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
W20260924 21:21:10.830147 10894 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:21:10.830240 10894 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:21:10.832612 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.825393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43046 (local address 127.20.115.253:35343)
0924 21:21:10.825428 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.825431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.825468 (+ 37us) server_negotiation.cc:407] Connection header received
0924 21:21:10.825537 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.825538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.825547 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.825601 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.826100 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.826562 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.827181 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.827397 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.827471 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.829807 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.829809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.829809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.829816 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.831460 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.831840 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.831841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.831841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.831900 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.832119 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.832120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.832120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.832246 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.832411 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.832450 (+ 39us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.832523 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260924 21:21:10.830997 10808 master_runner.cc:186] Error getting master registration for 127.20.115.254:44609: OK, instance_id { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" instance_seqno: 1790284870490112 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:21:10.833369 10808 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:21:10.833498 10726 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } attempt: 2
I20260924 21:21:10.834192 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:21:10.835407 10700 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f8aaa7536d824df5be091f2e07b67e1c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41712
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
I20260924 21:21:10.840859 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.833772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60776 (local address 127.20.115.252:36397)
0924 21:21:10.833808 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.833810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.833902 (+ 92us) server_negotiation.cc:407] Connection header received
0924 21:21:10.833922 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.833923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.833949 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.834012 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.834358 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.835052 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.835999 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.836336 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.836412 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.838126 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.838127 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.838128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.838134 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.839672 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.840057 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.840058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.840058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.840114 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.840316 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.840317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.840317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.840469 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.840539 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.840688 (+ 149us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.840762 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:21:10.840883 10728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.833722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60776)
0924 21:21:10.833805 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.833815 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:10.833852 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:10.834051 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:10.834052 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:10.834235 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:10.834274 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.834276 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.835144 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.835145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.835905 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.835908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.836439 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.836440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.836501 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.836943 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:10.836948 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:10.838033 (+ 1085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:10.839765 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.839766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.839766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.839977 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.840186 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.840187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.840187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.840246 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.840538 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:10.840539 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:10.840580 (+ 41us) client_negotiation.cc:769] Sending connection context
0924 21:21:10.840657 (+ 77us) client_negotiation.cc:240] Negotiation successful
0924 21:21:10.840746 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260924 21:21:10.842366 10726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c: Bootstrap starting.
I20260924 21:21:10.843075 10726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:10.843367 10726 log.cc:824] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:10.844213 10726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c: No bootstrap required, opened a new log
I20260924 21:21:10.845585 10726 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } }
I20260924 21:21:10.845719 10726 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8aaa7536d824df5be091f2e07b67e1c, State: Initialized, Role: FOLLOWER
I20260924 21:21:10.845796 10726 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } }
I20260924 21:21:10.846091 10726 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:10.846046 10897 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.846151 10700 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 21:21:10.846156 10897 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:10.846217 10700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:10.846882 10700 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } }
I20260924 21:21:10.847306 10850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8aaa7536d824df5be091f2e07b67e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9ae39f663cc040ddae69eb339debad4d"
I20260924 21:21:10.847437 10850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:10.848333 10850 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8aaa7536d824df5be091f2e07b67e1c in term 1.
I20260924 21:21:10.848536 10676 leader_election.cc:302] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ae39f663cc040ddae69eb339debad4d, f8aaa7536d824df5be091f2e07b67e1c; no voters:
I20260924 21:21:10.848649 10700 leader_election.cc:288] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [CANDIDATE]: Term 1 election: Requested vote from peers c1e2e22c345f40e0a9eb14e74fbcace6 (127.20.115.253:35343), 9ae39f663cc040ddae69eb339debad4d (127.20.115.252:36397)
I20260924 21:21:10.848697 10897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [term 1 FOLLOWER]: Leader election won for term 1
W20260924 21:21:10.850289 10908 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:21:10.850346 10908 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:21:10.848680 10658 master_runner.cc:186] Error getting master registration for 127.20.115.253:35343: OK, instance_id { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" instance_seqno: 1790284870635441 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:21:10.852339 10658 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:21:10.852496 10897 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [term 1 LEADER]: Becoming Leader. State: Replica: f8aaa7536d824df5be091f2e07b67e1c, State: Running, Role: LEADER
I20260924 21:21:10.852584 10897 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } }
I20260924 21:21:10.853013 10917 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8aaa7536d824df5be091f2e07b67e1c. Latest consensus state: current_term: 1 leader_uuid: "f8aaa7536d824df5be091f2e07b67e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.853186 10917 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:10.853427 10919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:21:10.853653 10919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:21:10.855243 10850 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [term 1 FOLLOWER]: Refusing update from remote peer f8aaa7536d824df5be091f2e07b67e1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:21:10.855610 10917 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:21:10.857050 10883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8aaa7536d824df5be091f2e07b67e1c. Latest consensus state: current_term: 1 leader_uuid: "f8aaa7536d824df5be091f2e07b67e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.857152 10883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:10.857867 10729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.850051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41716 (local address 127.20.115.254:44609)
0924 21:21:10.850130 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.850132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.850291 (+ 159us) server_negotiation.cc:407] Connection header received
0924 21:21:10.850491 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.850492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.850502 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.850552 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.851208 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.851691 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.852622 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.852913 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.852960 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.854550 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.854551 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.854552 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.854558 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.856232 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.856635 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.856636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.856637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.856708 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.856960 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.856961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.856961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.857091 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.857181 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.857402 (+ 221us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.857770 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260924 21:21:10.858987 10917 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8aaa7536d824df5be091f2e07b67e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.859074 10917 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:10.859473 10883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8aaa7536d824df5be091f2e07b67e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.859560 10883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:10.859815 10920 mvcc.cc:205] Tried to move back new op lower bound from 7333006831020666880 to 7333006831012913152. Current Snapshot: MvccSnapshot[applied={T|T < 7333006831020666880}]
I20260924 21:21:10.860363 10921 mvcc.cc:205] Tried to move back new op lower bound from 7333006831020666880 to 7333006831012913152. Current Snapshot: MvccSnapshot[applied={T|T < 7333006831020666880}]
I20260924 21:21:10.862951 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.849850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60790 (local address 127.20.115.252:36397)
0924 21:21:10.849888 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.849890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.853184 (+ 3294us) server_negotiation.cc:407] Connection header received
0924 21:21:10.853319 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.853320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.853329 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.853357 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.854467 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.855192 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.856683 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.857002 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.857040 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.859617 (+ 2577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.859619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.859619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.859625 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.861377 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.861824 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.861825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.861825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.861896 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.862294 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.862295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.862295 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.862425 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.862510 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.862746 (+ 236us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.862840 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260924 21:21:10.863646 10919 catalog_manager.cc:1347] Generated new cluster ID: 67d41d819b0741af94dd5f4a9b4c071c
I20260924 21:21:10.863692 10919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:21:10.868085 10805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.850089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43058 (local address 127.20.115.253:35343)
0924 21:21:10.850177 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.850178 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.856888 (+ 6710us) server_negotiation.cc:407] Connection header received
0924 21:21:10.856983 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.856984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.856995 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.857027 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.858181 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.858885 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.863665 (+ 4780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.863954 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.864017 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.865411 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.865412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.865412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.865419 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.867034 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.867368 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.867368 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.867369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.867414 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.867573 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.867573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.867574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.867691 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.867759 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.867931 (+ 172us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.867999 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260924 21:21:10.868240 10801 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } attempt: 3
Sep 24 21:21:10 dist-test-slave-cxpd krb5kdc[10645](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284870, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
I20260924 21:21:10.876088 10803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.868428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:36397 (local address 127.0.0.1:60802)
0924 21:21:10.868499 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:10.868508 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:10.868537 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:10.868758 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:10.868759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:10.868936 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:10.868974 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.868975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.869834 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.869835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.870606 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:10.870607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.871318 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.871318 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:10.871378 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.871831 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:10.871836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:10.873258 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:10.875088 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.875089 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.875090 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.875291 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.875492 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:10.875493 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:10.875493 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:10.875545 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:10.875801 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:10.875802 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:10.875841 (+ 39us) client_negotiation.cc:769] Sending connection context
0924 21:21:10.875899 (+ 58us) client_negotiation.cc:240] Negotiation successful
0924 21:21:10.875984 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260924 21:21:10.876438 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:10.868607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60802 (local address 127.20.115.252:36397)
0924 21:21:10.868648 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.868650 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.868656 (+ 6us) server_negotiation.cc:407] Connection header received
0924 21:21:10.868673 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.868674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.868682 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.868722 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.869051 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.869747 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.870922 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:10.871212 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:10.871268 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:10.873349 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:10.873350 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:10.873351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:10.873357 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:10.874986 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.875376 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.875377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.875377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.875428 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:10.875603 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:10.875603 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:10.875604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:10.875725 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:10.875776 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:10.876281 (+ 505us) server_negotiation.cc:299] Negotiation successful
0924 21:21:10.876348 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260924 21:21:10.877964 10801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6: Bootstrap starting.
I20260924 21:21:10.878707 10801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:10.879020 10801 log.cc:824] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:10.882215 10919 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:21:10.882972 10919 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:21:10.885390 10801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6: No bootstrap required, opened a new log
I20260924 21:21:10.887419 10801 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } }
I20260924 21:21:10.887575 10801 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e2e22c345f40e0a9eb14e74fbcace6, State: Initialized, Role: FOLLOWER
I20260924 21:21:10.887667 10801 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } }
I20260924 21:21:10.887995 10801 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:10.888033 10775 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f8aaa7536d824df5be091f2e07b67e1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6"
I20260924 21:21:10.888147 10775 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:10.888931 10775 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f8aaa7536d824df5be091f2e07b67e1c in term 1.
I20260924 21:21:10.889322 10774 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [term 1 FOLLOWER]: Refusing update from remote peer f8aaa7536d824df5be091f2e07b67e1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:21:10.889608 10917 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260924 21:21:10.893874 10937 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:21:10.893962 10937 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:21:10.895551 10919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c: Generated new TSK 0
I20260924 21:21:10.895761 10919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260924 21:21:10.896857 10926 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8aaa7536d824df5be091f2e07b67e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.896914 10938 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8aaa7536d824df5be091f2e07b67e1c. Latest consensus state: current_term: 1 leader_uuid: "f8aaa7536d824df5be091f2e07b67e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.897046 10938 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:10.897106 10938 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f8aaa7536d824df5be091f2e07b67e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.897217 10938 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:10.897686 10939 mvcc.cc:205] Tried to move back new op lower bound from 7333006831126798336 to 7333006831012913152. Current Snapshot: MvccSnapshot[applied={T|T < 7333006831020666880}]
I20260924 21:21:10.898419 10897 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f8aaa7536d824df5be091f2e07b67e1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8aaa7536d824df5be091f2e07b67e1c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 44609 } } peers { permanent_uuid: "c1e2e22c345f40e0a9eb14e74fbcace6" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35343 } } peers { permanent_uuid: "9ae39f663cc040ddae69eb339debad4d" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 36397 } } }
I20260924 21:21:10.898506 10897 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8aaa7536d824df5be091f2e07b67e1c [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:10.896950 10926 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6 [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:21:10.929935 10881 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:10.928270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.20.115.252:36397)
0924 21:21:10.928315 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.928319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.929169 (+ 850us) server_negotiation.cc:407] Connection header received
0924 21:21:10.929281 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.929282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.929293 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.929324 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.929867 (+ 543us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60808: BlockingRecv error: recv got EOF from 127.0.0.1:60808 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 21:21:10.930835 10729 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:10.928719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41730 (local address 127.20.115.254:44609)
0924 21:21:10.928758 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.928759 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.930310 (+ 1551us) server_negotiation.cc:407] Connection header received
0924 21:21:10.930407 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.930408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.930419 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.930447 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.930773 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41730: BlockingRecv error: recv got EOF from 127.0.0.1:41730 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 21:21:10.931694 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:10.929067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43068 (local address 127.20.115.253:35343)
0924 21:21:10.929187 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.929189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.931149 (+ 1960us) server_negotiation.cc:407] Connection header received
0924 21:21:10.931279 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.931280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.931291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.931319 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.931603 (+ 284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43068: BlockingRecv error: recv got EOF from 127.0.0.1:43068 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260924 21:21:10.933784 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:10.933048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.20.115.253:35343)
0924 21:21:10.933082 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.933084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.933282 (+ 198us) server_negotiation.cc:407] Connection header received
0924 21:21:10.933397 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.933398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.933415 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.933443 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.933741 (+ 298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43078: BlockingRecv error: recv got EOF from 127.0.0.1:43078 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 21:21:10.936818 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:10.936097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43080 (local address 127.20.115.253:35343)
0924 21:21:10.936128 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.936129 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.936328 (+ 199us) server_negotiation.cc:407] Connection header received
0924 21:21:10.936419 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.936420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.936430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.936458 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.936754 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43080: BlockingRecv error: recv got EOF from 127.0.0.1:43080 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260924 21:21:10.940659 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:10.939976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43086 (local address 127.20.115.253:35343)
0924 21:21:10.940010 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.940012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.940205 (+ 193us) server_negotiation.cc:407] Connection header received
0924 21:21:10.940292 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.940293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.940302 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.940330 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.940610 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43086: BlockingRecv error: recv got EOF from 127.0.0.1:43086 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 21:21:10.946717 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:10.945912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43100 (local address 127.20.115.253:35343)
0924 21:21:10.945944 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.945946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.946204 (+ 258us) server_negotiation.cc:407] Connection header received
0924 21:21:10.946296 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.946297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.946305 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.946335 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.946674 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43100: BlockingRecv error: recv got EOF from 127.0.0.1:43100 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 21:21:10.956727 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:10.955985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43114 (local address 127.20.115.253:35343)
0924 21:21:10.956028 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.956030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.956231 (+ 201us) server_negotiation.cc:407] Connection header received
0924 21:21:10.956324 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.956325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.956334 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.956362 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.956682 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43114: BlockingRecv error: recv got EOF from 127.0.0.1:43114 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260924 21:21:10.975041 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:10.974186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43122 (local address 127.20.115.253:35343)
0924 21:21:10.974235 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:10.974237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:10.974466 (+ 229us) server_negotiation.cc:407] Connection header received
0924 21:21:10.974566 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:10.974567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:10.974577 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:10.974605 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:10.974958 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43122: BlockingRecv error: recv got EOF from 127.0.0.1:43122 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260924 21:21:11.009501 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:11.008564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43132 (local address 127.20.115.253:35343)
0924 21:21:11.008609 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:11.008625 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:11.008909 (+ 284us) server_negotiation.cc:407] Connection header received
0924 21:21:11.009020 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:11.009021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:11.009033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:11.009064 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:11.009441 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43132: BlockingRecv error: recv got EOF from 127.0.0.1:43132 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 21:21:11.076135 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:11.075252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.20.115.253:35343)
0924 21:21:11.075321 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:11.075323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:11.075504 (+ 181us) server_negotiation.cc:407] Connection header received
0924 21:21:11.075589 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:11.075590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:11.075609 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:11.075666 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:11.076019 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43140: BlockingRecv error: recv got EOF from 127.0.0.1:43140 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260924 21:21:11.206214 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:11.205372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.20.115.253:35343)
0924 21:21:11.205460 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:11.205462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:11.205650 (+ 188us) server_negotiation.cc:407] Connection header received
0924 21:21:11.205681 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:11.205682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:11.205693 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:11.205740 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:11.206117 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43156: BlockingRecv error: recv got EOF from 127.0.0.1:43156 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260924 21:21:11.464195 10805 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:11.463397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43170 (local address 127.20.115.253:35343)
0924 21:21:11.463473 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:11.463475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:11.463547 (+ 72us) server_negotiation.cc:407] Connection header received
0924 21:21:11.463599 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:11.463600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:11.463610 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:11.463665 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:11.464064 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43170: BlockingRecv error: recv got EOF from 127.0.0.1:43170 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260924 21:21:11.830679 10894 catalog_manager.cc:1259] Loaded cluster ID: 67d41d819b0741af94dd5f4a9b4c071c
I20260924 21:21:11.830780 10894 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d: loading cluster ID for follower catalog manager: success
I20260924 21:21:11.831545 10894 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d: acquiring CA information for follower catalog manager: success
I20260924 21:21:11.831845 10894 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9ae39f663cc040ddae69eb339debad4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:21:11.894441 10937 catalog_manager.cc:1259] Loaded cluster ID: 67d41d819b0741af94dd5f4a9b4c071c
I20260924 21:21:11.894521 10937 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6: loading cluster ID for follower catalog manager: success
I20260924 21:21:11.895381 10937 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6: acquiring CA information for follower catalog manager: success
I20260924 21:21:11.895792 10937 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c1e2e22c345f40e0a9eb14e74fbcace6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:21:11.979872 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:11.977289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43182 (local address 127.20.115.253:35343)
0924 21:21:11.977424 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:11.977427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:11.977438 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:21:11.977505 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:11.977506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:11.977556 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:11.977635 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:11.977803 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.978291 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.979050 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.979259 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.979322 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:11.979497 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:11.979613 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:11.979687 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:11.979780 (+ 93us) server_negotiation.cc:299] Negotiation successful
0924 21:21:11.979801 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260924 21:21:11.983901 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:11.981848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60824 (local address 127.20.115.252:36397)
0924 21:21:11.981941 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:11.981943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:11.981952 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:21:11.982015 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:11.982016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:11.982047 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:11.982104 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:11.982206 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.982662 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.983325 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.983504 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.983552 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:11.983662 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:11.983748 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:11.983796 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:11.983829 (+ 33us) server_negotiation.cc:299] Negotiation successful
0924 21:21:11.983843 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74,"thread_start_us":53,"threads_started":1}
I20260924 21:21:11.988389 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:11.985459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60834 (local address 127.20.115.252:36397)
0924 21:21:11.985499 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:11.985501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:11.985686 (+ 185us) server_negotiation.cc:407] Connection header received
0924 21:21:11.985879 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:11.985880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:11.985888 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:11.985934 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:11.986074 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.986768 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.987599 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.987876 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.987914 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:11.988147 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:11.988174 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:11.988202 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:11.988312 (+ 110us) server_negotiation.cc:299] Negotiation successful
0924 21:21:11.988330 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:21:11.988394 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:11.985423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41744 (local address 127.20.115.254:44609)
0924 21:21:11.985588 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:11.985590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:11.985608 (+ 18us) server_negotiation.cc:407] Connection header received
0924 21:21:11.985696 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:11.985696 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:11.985745 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:11.985848 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:11.985976 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.986472 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.987511 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.987819 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.987883 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:11.988062 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:11.988161 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:11.988225 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:11.988286 (+ 61us) server_negotiation.cc:299] Negotiation successful
0924 21:21:11.988313 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":46,"threads_started":1}
I20260924 21:21:11.988583 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:11.985582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.20.115.253:35343)
0924 21:21:11.985619 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:11.985621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:11.985712 (+ 91us) server_negotiation.cc:407] Connection header received
0924 21:21:11.985768 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:11.985769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:11.985779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:11.985829 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:11.985985 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.986717 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.987748 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:11.988035 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:11.988096 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:11.988279 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:11.988311 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:11.988339 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:11.988456 (+ 117us) server_negotiation.cc:299] Negotiation successful
0924 21:21:11.988480 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:21:11.990131 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 10658
I20260924 21:21:11.995349 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 10733
I20260924 21:21:12.000283 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 10808
2026-09-24T21:21:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2774 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 24 21:21:12 dist-test-slave-cxpd krb5kdc[11027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 24 21:21:12 dist-test-slave-cxpd krb5kdc[11027](info): set up 2 sockets
Sep 24 21:21:12 dist-test-slave-cxpd krb5kdc[11027](info): commencing operation
krb5kdc: starting...
W20260924 21:21:13.121147 20943 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/test-user.keytab.
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-24T21:21:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-24T21:21:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.115.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
Entry for principal kudu/127.20.115.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab.
I20260924 21:21:13.243835 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43267
--webserver_interface=127.20.115.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:43267,127.20.115.253:35035,127.20.115.252:39701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260924 21:21:13.322726 11040 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:21:13.322975 11040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:13.323005 11040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:13.323076 11040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:13.324249 11040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:13.324317 11040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:13.324342 11040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:13.324363 11040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:13.324391 11040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:13.325892 11040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:43267,127.20.115.253:35035,127.20.115.252:39701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.254.keytab
--principal=kudu/127.20.115.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.254:43267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.115.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:13.326175 11040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:13.326408 11040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:13.331336 11046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:13.331370 11048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:13.331350 11045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:13.331511 11040 server_base.cc:1034] running on GCE node
I20260924 21:21:13.331835 11040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:13.332126 11040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:13.333333 11040 hybrid_clock.cc:648] HybridClock initialized: now 1790284873333308 us; error 36 us; skew 500 ppm
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:13.335844 11040 init.cc:377] Logged in from keytab as kudu/127.20.115.254@KRBTEST.COM (short username kudu)
I20260924 21:21:13.336288 11040 webserver.cc:460] Webserver started at http://127.20.115.254:44211/ using document root <none> and password file <none>
I20260924 21:21:13.336532 11040 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:13.336603 11040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:13.336755 11040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:13.337885 11040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/data/instance:
uuid: "6a8e38fdfafc49bbb3152bccc76f788c"
format_stamp: "Formatted at 2026-09-24 21:21:13 on dist-test-slave-cxpd"
server_key: "e6e46e7b4034427dd36051986c88906d"
server_key_iv: "1bfab40c7311e4a017dc4573185d52b4"
server_key_version: "encryptionkey@0"
I20260924 21:21:13.338243 11040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/wal/instance:
uuid: "6a8e38fdfafc49bbb3152bccc76f788c"
format_stamp: "Formatted at 2026-09-24 21:21:13 on dist-test-slave-cxpd"
server_key: "e6e46e7b4034427dd36051986c88906d"
server_key_iv: "1bfab40c7311e4a017dc4573185d52b4"
server_key_version: "encryptionkey@0"
I20260924 21:21:13.339670 11040 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260924 21:21:13.340427 11055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:13.340643 11040 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260924 21:21:13.340754 11040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/wal
uuid: "6a8e38fdfafc49bbb3152bccc76f788c"
format_stamp: "Formatted at 2026-09-24 21:21:13 on dist-test-slave-cxpd"
server_key: "e6e46e7b4034427dd36051986c88906d"
server_key_iv: "1bfab40c7311e4a017dc4573185d52b4"
server_key_version: "encryptionkey@0"
I20260924 21:21:13.340894 11040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:13.356458 11040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:13.358978 11040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:13.359124 11040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:13.363044 11040 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.254:43267
I20260924 21:21:13.363127 11107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.254:43267 every 8 connection(s)
I20260924 21:21:13.363458 11040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/data/info.pb
I20260924 21:21:13.364322 11108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:13.365891 11108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:13.368219 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 11040
I20260924 21:21:13.368309 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-0/wal/instance
I20260924 21:21:13.368507 20943 external_mini_cluster.cc:1428] Setting key ccce44516a1e6857f94a7bb246a2ba47
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:21:13.376632 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.366597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43267 (local address 127.0.0.1:35148)
0924 21:21:13.366792 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.366834 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:13.367317 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:13.367600 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:13.367602 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:13.367872 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:13.368034 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.368040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.369208 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.369209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.370191 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.370194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.370897 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.370897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.370953 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.371466 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:13.371474 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:13.373126 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:13.375237 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.375243 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.375244 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.375435 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.375695 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.375696 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.375696 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.375772 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.376236 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:13.376240 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:13.376292 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 21:21:13.376364 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:21:13.376481 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":102,"threads_started":1}
I20260924 21:21:13.376672 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.366600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.20.115.254:43267)
0924 21:21:13.366801 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.366810 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.366934 (+ 124us) server_negotiation.cc:407] Connection header received
0924 21:21:13.367470 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.367472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.367491 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.367595 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.368139 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.369118 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.370277 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.370806 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.370929 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.373252 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.373261 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.373261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.373269 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.375145 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.375517 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.375520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.375521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.375620 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.375839 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.375841 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.375841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.376150 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.376248 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.376428 (+ 180us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.376565 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":156,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.20.115.253@KRBTEST.COM; defaulting to no policy
I20260924 21:21:13.377455 11108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:13.377831 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.377727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35035 (local address 127.0.0.1:41554)
0924 21:21:13.377774 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.377814 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:21:13.378126 11059 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
I20260924 21:21:13.378544 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.378462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35035 (local address 127.0.0.1:41562)
0924 21:21:13.378505 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.378529 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:21:13.378671 11108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.253:35035: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab.
Entry for principal kudu/127.20.115.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab.
I20260924 21:21:13.395934 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.253:35035
--webserver_interface=127.20.115.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab
--principal=kudu/127.20.115.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:43267,127.20.115.253:35035,127.20.115.252:39701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:21:13.427883 11108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } attempt: 1
I20260924 21:21:13.428341 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.428214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35035 (local address 127.0.0.1:41566)
0924 21:21:13.428275 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.428315 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:21:13.428750 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.428660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35035 (local address 127.0.0.1:41580)
0924 21:21:13.428708 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.428734 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260924 21:21:13.428865 11108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.253:35035: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
W20260924 21:21:13.476732 11115 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:21:13.476969 11115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:13.476987 11115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:13.477007 11115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:13.478122 11115 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:13.478157 11115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:13.478171 11115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:13.478183 11115 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:13.478195 11115 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:13.479580 11115 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:43267,127.20.115.253:35035,127.20.115.252:39701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.253.keytab
--principal=kudu/127.20.115.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.253:35035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.115.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11115
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:13.479775 11115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:13.479938 11115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:13.484596 11120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:13.484702 11121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:13.484726 11115 server_base.cc:1034] running on GCE node
W20260924 21:21:13.484833 11123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:13.485052 11115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:13.485340 11115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:13.486485 11115 hybrid_clock.cc:648] HybridClock initialized: now 1790284873486467 us; error 40 us; skew 500 ppm
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:13.488906 11115 init.cc:377] Logged in from keytab as kudu/127.20.115.253@KRBTEST.COM (short username kudu)
I20260924 21:21:13.489342 11115 webserver.cc:460] Webserver started at http://127.20.115.253:39873/ using document root <none> and password file <none>
I20260924 21:21:13.489516 11115 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:13.489554 11115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:13.489647 11115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:13.490609 11115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/data/instance:
uuid: "2e302bf883c4437c82cf24753620849d"
format_stamp: "Formatted at 2026-09-24 21:21:13 on dist-test-slave-cxpd"
server_key: "b50d706a6a369cd322e704e69783214c"
server_key_iv: "0377845ccf4ae8f83e479d5a1dcc2a32"
server_key_version: "encryptionkey@0"
I20260924 21:21:13.490913 11115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/wal/instance:
uuid: "2e302bf883c4437c82cf24753620849d"
format_stamp: "Formatted at 2026-09-24 21:21:13 on dist-test-slave-cxpd"
server_key: "b50d706a6a369cd322e704e69783214c"
server_key_iv: "0377845ccf4ae8f83e479d5a1dcc2a32"
server_key_version: "encryptionkey@0"
I20260924 21:21:13.492125 11115 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260924 21:21:13.492817 11130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:13.493007 11115 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:21:13.493104 11115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/wal
uuid: "2e302bf883c4437c82cf24753620849d"
format_stamp: "Formatted at 2026-09-24 21:21:13 on dist-test-slave-cxpd"
server_key: "b50d706a6a369cd322e704e69783214c"
server_key_iv: "0377845ccf4ae8f83e479d5a1dcc2a32"
server_key_version: "encryptionkey@0"
I20260924 21:21:13.493194 11115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:13.497020 11108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } attempt: 2
I20260924 21:21:13.497414 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.497287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35035 (local address 127.0.0.1:41596)
0924 21:21:13.497343 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.497379 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:21:13.497763 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.497668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35035 (local address 127.0.0.1:41612)
0924 21:21:13.497714 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.497740 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 21:21:13.497871 11108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.253:35035: Network error: Client connection negotiation failed: client connection to 127.20.115.253:35035: connect: Connection refused (error 111)
I20260924 21:21:13.504426 11115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:13.506245 11115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:13.506349 11115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:13.510084 11115 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.253:35035
I20260924 21:21:13.510131 11182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.253:35035 every 8 connection(s)
I20260924 21:21:13.510468 11115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/data/info.pb
I20260924 21:21:13.511323 11183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:13.512856 11183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:21:13.520587 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 11115
I20260924 21:21:13.520711 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-1/wal/instance
I20260924 21:21:13.520838 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.513513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.20.115.254:43267)
0924 21:21:13.513578 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.513580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.513774 (+ 194us) server_negotiation.cc:407] Connection header received
0924 21:21:13.514216 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.514218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.514230 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.514285 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.514818 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.515295 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.516188 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.516375 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.516424 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.518368 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.518369 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.518370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.518376 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.519569 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.519917 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.519918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.519918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.519966 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.520177 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.520178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.520178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.520313 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.520364 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.520616 (+ 252us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.520722 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260924 21:21:13.520993 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.513486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43267 (local address 127.0.0.1:35152)
0924 21:21:13.513653 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.513693 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:13.514129 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:13.514333 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:13.514336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:13.514570 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:13.514749 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.514752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.515372 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.515373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.516112 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.516113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.516443 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.516444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.516537 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.516995 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:13.517003 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:13.518289 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:13.519643 (+ 1354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.519649 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.519651 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.519853 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.520023 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.520024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.520024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.520115 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.520370 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:13.520374 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:13.520476 (+ 102us) client_negotiation.cc:769] Sending connection context
0924 21:21:13.520589 (+ 113us) client_negotiation.cc:240] Negotiation successful
0924 21:21:13.520737 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260924 21:21:13.521001 20943 external_mini_cluster.cc:1428] Setting key 9f275a40401cb6f908cd2eccbda90b66
I20260924 21:21:13.521589 11183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.115.252@KRBTEST.COM; defaulting to no policy
I20260924 21:21:13.528803 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.521843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35035 (local address 127.0.0.1:41620)
0924 21:21:13.521895 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.521906 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:13.521944 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:13.522206 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:13.522207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:13.522339 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:13.522378 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.522379 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.523208 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.523209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.523703 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.523704 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.524190 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.524191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.524252 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.524657 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:13.524660 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:13.526122 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:13.527752 (+ 1630us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.527753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.527754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.527916 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.528125 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.528126 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.528126 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.528179 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.528460 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:13.528461 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:13.528497 (+ 36us) client_negotiation.cc:769] Sending connection context
0924 21:21:13.528571 (+ 74us) client_negotiation.cc:240] Negotiation successful
0924 21:21:13.528674 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260924 21:21:13.528882 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.521915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41620 (local address 127.20.115.253:35035)
0924 21:21:13.522056 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.522060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.522081 (+ 21us) server_negotiation.cc:407] Connection header received
0924 21:21:13.522097 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.522098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.522108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.522206 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.522457 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.523119 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.523797 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.524101 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.524190 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.526208 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.526214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.526214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.526220 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.527653 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.528007 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.528009 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.528010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.528061 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.528260 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.528261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.528261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.528393 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.528483 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.528603 (+ 120us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.528727 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260924 21:21:13.529733 11183 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:13.530025 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.529927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44258)
0924 21:21:13.529970 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.530009 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 21:21:13.530154 11131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
I20260924 21:21:13.530520 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.530450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44274)
0924 21:21:13.530475 (+ 25us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.530502 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260924 21:21:13.530645 11183 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:39701: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.115.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.115.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab.
Entry for principal kudu/127.20.115.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab.
I20260924 21:21:13.545050 20943 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskNZGLg7/build/release/bin/kudu
/tmp/dist-test-taskNZGLg7/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.252:39701
--webserver_interface=127.20.115.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab
--principal=kudu/127.20.115.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.115.212:45099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.115.254:43267,127.20.115.253:35035,127.20.115.252:39701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260924 21:21:13.579878 11183 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } attempt: 1
I20260924 21:21:13.580417 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.580271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44278)
0924 21:21:13.580344 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.580386 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260924 21:21:13.580816 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.580683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44292)
0924 21:21:13.580728 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.580758 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260924 21:21:13.580912 11183 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:39701: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
I20260924 21:21:13.589047 11108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } attempt: 3
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
I20260924 21:21:13.596329 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.589450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35035 (local address 127.0.0.1:41636)
0924 21:21:13.589512 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.589523 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:13.589565 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:13.589720 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:13.589721 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:13.589909 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:13.589985 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.589987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.590637 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.590638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.591447 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.591450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.591795 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.591796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.591872 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.592320 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:13.592325 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:13.593648 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:13.595241 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.595242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.595242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.595416 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.595649 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.595650 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.595650 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.595715 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.595993 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:13.595994 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:13.596045 (+ 51us) client_negotiation.cc:769] Sending connection context
0924 21:21:13.596157 (+ 112us) client_negotiation.cc:240] Negotiation successful
0924 21:21:13.596259 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260924 21:21:13.596400 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.589512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41636 (local address 127.20.115.253:35035)
0924 21:21:13.589571 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.589573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.589586 (+ 13us) server_negotiation.cc:407] Connection header received
0924 21:21:13.589613 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.589614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.589622 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.589684 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.590059 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.590562 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.591528 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.591726 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.591776 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.593742 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.593745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.593745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.593752 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.595159 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.595501 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.595503 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.595503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.595562 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.595784 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.595785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.595786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.595922 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.596004 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.596214 (+ 210us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.596307 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260924 21:21:13.596861 11108 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:13.597216 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.597106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44304)
0924 21:21:13.597162 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.597189 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260924 21:21:13.597538 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.597444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44318)
0924 21:21:13.597495 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.597518 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260924 21:21:13.597628 11108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:39701: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
W20260924 21:21:13.626543 11190 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260924 21:21:13.626770 11190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260924 21:21:13.626798 11190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260924 21:21:13.626854 11190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260924 21:21:13.627990 11190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260924 21:21:13.628051 11190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260924 21:21:13.628077 11190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260924 21:21:13.628106 11190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260924 21:21:13.628135 11190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260924 21:21:13.628759 11108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } attempt: 1
I20260924 21:21:13.629240 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.629081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44322)
0924 21:21:13.629184 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.629213 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260924 21:21:13.629540 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.629451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44332)
0924 21:21:13.629494 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.629516 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260924 21:21:13.629619 11108 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:39701: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
I20260924 21:21:13.629873 11190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.115.212:45099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.115.254:43267,127.20.115.253:35035,127.20.115.252:39701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/krb5kdc/kudu_127.20.115.252.keytab
--principal=kudu/127.20.115.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.115.252:39701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.115.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 24 Sep 2026 19:49:06 UTC on ccycloud.quasar-vhqjoc.root.comops.site
build id 4670864
I20260924 21:21:13.630149 11190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260924 21:21:13.630419 11190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260924 21:21:13.635002 11196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:13.635109 11195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260924 21:21:13.635072 11198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260924 21:21:13.635319 11190 server_base.cc:1034] running on GCE node
I20260924 21:21:13.635543 11190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260924 21:21:13.635816 11190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260924 21:21:13.636971 11190 hybrid_clock.cc:648] HybridClock initialized: now 1790284873636968 us; error 36 us; skew 500 ppm
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260924 21:21:13.639436 11190 init.cc:377] Logged in from keytab as kudu/127.20.115.252@KRBTEST.COM (short username kudu)
I20260924 21:21:13.639873 11190 webserver.cc:460] Webserver started at http://127.20.115.252:32981/ using document root <none> and password file <none>
I20260924 21:21:13.640098 11190 fs_manager.cc:359] Metadata directory not provided
I20260924 21:21:13.640162 11190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260924 21:21:13.640303 11190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260924 21:21:13.641381 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/data/instance:
uuid: "12ee677e0f054626bc486dc984435c3b"
format_stamp: "Formatted at 2026-09-24 21:21:13 on dist-test-slave-cxpd"
server_key: "293c7f564778b9480e2d06301172164c"
server_key_iv: "c64f491bc682bb9651c6397831746f17"
server_key_version: "encryptionkey@0"
I20260924 21:21:13.641728 11190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/wal/instance:
uuid: "12ee677e0f054626bc486dc984435c3b"
format_stamp: "Formatted at 2026-09-24 21:21:13 on dist-test-slave-cxpd"
server_key: "293c7f564778b9480e2d06301172164c"
server_key_iv: "c64f491bc682bb9651c6397831746f17"
server_key_version: "encryptionkey@0"
I20260924 21:21:13.643028 11190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260924 21:21:13.643806 11205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260924 21:21:13.644004 11190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260924 21:21:13.644107 11190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/data,/tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/wal
uuid: "12ee677e0f054626bc486dc984435c3b"
format_stamp: "Formatted at 2026-09-24 21:21:13 on dist-test-slave-cxpd"
server_key: "293c7f564778b9480e2d06301172164c"
server_key_iv: "c64f491bc682bb9651c6397831746f17"
server_key_version: "encryptionkey@0"
I20260924 21:21:13.644186 11190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260924 21:21:13.649039 11183 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } attempt: 2
I20260924 21:21:13.649434 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.649324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44346)
0924 21:21:13.649375 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.649409 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260924 21:21:13.649791 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.649691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44352)
0924 21:21:13.649739 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.649770 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260924 21:21:13.649874 11183 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.115.252:39701: Network error: Client connection negotiation failed: client connection to 127.20.115.252:39701: connect: Connection refused (error 111)
I20260924 21:21:13.655319 11190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260924 21:21:13.657682 11190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260924 21:21:13.657842 11190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260924 21:21:13.661876 11190 rpc_server.cc:307] RPC server started. Bound to: 127.20.115.252:39701
I20260924 21:21:13.661908 11257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.115.252:39701 every 8 connection(s)
I20260924 21:21:13.662264 11190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/data/info.pb
I20260924 21:21:13.663161 11258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260924 21:21:13.664659 11258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } has no permanent_uuid. Determining permanent_uuid...
I20260924 21:21:13.669325 20943 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskNZGLg7/build/release/bin/kudu as pid 11190
I20260924 21:21:13.669423 20943 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskNZGLg7/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790284606872664-20943-0/minicluster-data/master-2/wal/instance
I20260924 21:21:13.669638 20943 external_mini_cluster.cc:1428] Setting key 0316557c6d52936224072c1a3b583c66
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
I20260924 21:21:13.674470 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.665393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35154 (local address 127.20.115.254:43267)
0924 21:21:13.665473 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.665475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.665656 (+ 181us) server_negotiation.cc:407] Connection header received
0924 21:21:13.666086 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.666087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.666096 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.666150 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.666750 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.667276 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.668197 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.668409 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.668495 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.671321 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.671323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.671324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.671333 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.672755 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.673307 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.673308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.673309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.673376 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.673651 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.673652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.673652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.673794 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.673862 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.674290 (+ 428us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.674390 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260924 21:21:13.674613 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.665358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.254:43267 (local address 127.0.0.1:35154)
0924 21:21:13.665530 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.665566 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:13.666000 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:13.666199 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:13.666201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:13.666461 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:13.666644 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.666648 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.667362 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.667363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.668114 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.668115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.668524 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.668526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.668640 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.669256 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:13.669271 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:13.671238 (+ 1967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:13.672907 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.672914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.672916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.673233 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.673469 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.673470 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.673470 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.673577 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.673881 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:13.673887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:13.674051 (+ 164us) client_negotiation.cc:769] Sending connection context
0924 21:21:13.674198 (+ 147us) client_negotiation.cc:240] Negotiation successful
0924 21:21:13.674347 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260924 21:21:13.675242 11258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
I20260924 21:21:13.679957 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.670085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.20.115.252:39701)
0924 21:21:13.670221 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.670225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.670296 (+ 71us) server_negotiation.cc:407] Connection header received
0924 21:21:13.670357 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.670359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.670374 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.670439 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.671211 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.672316 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.673307 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.673770 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.673846 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.676379 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.676386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.676387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.676394 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.678218 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.678989 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.678993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.678994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.679065 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.679320 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.679321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.679321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.679507 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.679694 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.679828 (+ 134us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.679889 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":85,"threads_started":1}
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
I20260924 21:21:13.683566 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.675514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.253:35035 (local address 127.0.0.1:41640)
0924 21:21:13.675563 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.675573 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:13.675614 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:13.676281 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:13.676282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:13.676439 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:13.676493 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.676495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.677354 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.677356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.678212 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.678215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.678587 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.678588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.678658 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.679028 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:13.679032 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:13.680434 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:13.682542 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.682542 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.682543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.682720 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.682934 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.682935 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.682935 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.683006 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.683304 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:13.683305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:13.683343 (+ 38us) client_negotiation.cc:769] Sending connection context
0924 21:21:13.683409 (+ 66us) client_negotiation.cc:240] Negotiation successful
0924 21:21:13.683481 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:21:13.683625 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.676101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.20.115.253:35035)
0924 21:21:13.676157 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.676159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.676167 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:21:13.676191 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.676191 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.676203 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.676277 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.676580 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.677249 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.678306 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.678499 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.678558 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.680513 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.680514 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.680515 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.680522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.682463 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.682806 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.682808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.682808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.682867 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.683071 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.683072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.683072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.683215 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.683285 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.683462 (+ 177us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.683534 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260924 21:21:13.683995 11258 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } has no permanent_uuid. Determining permanent_uuid...
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.254@KRBTEST.COM
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.252@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
I20260924 21:21:13.689297 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.681160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35160 (local address 127.20.115.254:43267)
0924 21:21:13.681252 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.681254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.681585 (+ 331us) server_negotiation.cc:407] Connection header received
0924 21:21:13.681652 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.681653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.681665 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.681727 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.682334 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.682796 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.683741 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.684017 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.684063 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.686488 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.686490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.686490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.686500 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.688082 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.688447 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.688448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.688448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.688504 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.688705 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.688706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.688707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.688885 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.689088 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.689124 (+ 36us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.689198 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260924 21:21:13.690847 11260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.684260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44372)
0924 21:21:13.684332 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.684341 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:13.684383 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:13.684529 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:13.684530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:13.684673 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:13.684711 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.684713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.685391 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.685392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.685922 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.685924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.686304 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.686305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.686365 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.686752 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:13.686756 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:13.688019 (+ 1263us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:13.689836 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.689837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.689837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.690030 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.690229 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.690230 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.690230 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.690283 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.690516 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:13.690517 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:13.690569 (+ 52us) client_negotiation.cc:769] Sending connection context
0924 21:21:13.690651 (+ 82us) client_negotiation.cc:240] Negotiation successful
0924 21:21:13.690750 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260924 21:21:13.691069 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.684296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.20.115.252:39701)
0924 21:21:13.684326 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.684328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.684375 (+ 47us) server_negotiation.cc:407] Connection header received
0924 21:21:13.684429 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.684430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.684441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.684509 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.684864 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.685306 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.686006 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.686228 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.686288 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.688119 (+ 1831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.688120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.688120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.688127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.689749 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.690109 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.690109 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.690110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.690159 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.690340 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.690341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.690341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.690449 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.690497 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.690668 (+ 171us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.690989 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":246,"server-negotiator.queue_time_us":16}
I20260924 21:21:13.692356 11258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b: Bootstrap starting.
I20260924 21:21:13.693302 11258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:13.693591 11258 log.cc:824] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b: Log is configured to *not* fsync() on all Append() calls
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.115.253@KRBTEST.COM
I20260924 21:21:13.694868 11258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b: No bootstrap required, opened a new log
I20260924 21:21:13.696532 11258 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } }
I20260924 21:21:13.696676 11258 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12ee677e0f054626bc486dc984435c3b, State: Initialized, Role: FOLLOWER
I20260924 21:21:13.696767 11258 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } }
I20260924 21:21:13.697122 11258 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:13.697105 11264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.697259 11264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:13.697796 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.690011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41644 (local address 127.20.115.253:35035)
0924 21:21:13.690070 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.690072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.690081 (+ 9us) server_negotiation.cc:407] Connection header received
0924 21:21:13.690138 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.690139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.690147 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.690190 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.690758 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.691418 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.692073 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.692292 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.692368 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.694755 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.694757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.694757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.694772 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.696326 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.696673 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.696674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.696674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.696734 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.697024 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.697025 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.697026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.697173 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.697353 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.697561 (+ 208us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.697654 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:21:13.699016 11275 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:21:13.699071 11275 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:21:13.699622 20943 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260924 21:21:13.700503 11082 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6a8e38fdfafc49bbb3152bccc76f788c"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35160
I20260924 21:21:13.699381 11190 master_runner.cc:186] Error getting master registration for 127.20.115.254:43267: OK, instance_id { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" instance_seqno: 1790284873361985 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:21:13.702178 11190 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:21:13.704759 11108 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } attempt: 2
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.254@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
I20260924 21:21:13.712918 11111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.705063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44376)
0924 21:21:13.705114 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.705124 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:13.705164 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:13.705319 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:13.705320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:13.705506 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:13.705557 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.705560 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.706450 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.706452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.707276 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.707278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.707759 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.707760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.707845 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.708342 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:13.708348 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:13.709963 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:13.711762 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.711763 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.711764 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.711983 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.712190 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.712190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.712191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.712253 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.712573 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:13.712574 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:13.712621 (+ 47us) client_negotiation.cc:769] Sending connection context
0924 21:21:13.712688 (+ 67us) client_negotiation.cc:240] Negotiation successful
0924 21:21:13.712812 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260924 21:21:13.712940 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.705117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.20.115.252:39701)
0924 21:21:13.705159 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.705161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.705169 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:21:13.705233 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.705233 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.705244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.705296 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.705643 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.706359 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.707362 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.707672 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.707744 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.710051 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.710052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.710053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.710060 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.711678 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.712058 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.712059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.712060 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.712121 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.712320 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.712321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.712321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.712504 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.712576 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.712729 (+ 153us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.712850 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260924 21:21:13.714393 11108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c: Bootstrap starting.
I20260924 21:21:13.715036 11108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:13.715330 11108 log.cc:824] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:13.716161 11108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c: No bootstrap required, opened a new log
I20260924 21:21:13.717566 11108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } }
I20260924 21:21:13.717715 11108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a8e38fdfafc49bbb3152bccc76f788c, State: Initialized, Role: FOLLOWER
I20260924 21:21:13.717805 11108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } }
I20260924 21:21:13.718124 11108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:13.718159 11082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260924 21:21:13.718201 11082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:13.719041 11082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } }
I20260924 21:21:13.719327 11082 leader_election.cc:288] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [CANDIDATE]: Term 1 election: Requested vote from peers 2e302bf883c4437c82cf24753620849d (127.20.115.253:35035), 12ee677e0f054626bc486dc984435c3b (127.20.115.252:39701)
W20260924 21:21:13.720170 11289 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260924 21:21:13.720103 11232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "12ee677e0f054626bc486dc984435c3b"
W20260924 21:21:13.720216 11289 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:21:13.720245 11232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:13.721261 11232 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8e38fdfafc49bbb3152bccc76f788c in term 1.
I20260924 21:21:13.721535 11056 leader_election.cc:302] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12ee677e0f054626bc486dc984435c3b, 6a8e38fdfafc49bbb3152bccc76f788c; no voters:
I20260924 21:21:13.718097 11278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.722007 11278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:13.722165 11295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [term 1 FOLLOWER]: Leader election won for term 1
I20260924 21:21:13.722261 11295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [term 1 LEADER]: Becoming Leader. State: Replica: 6a8e38fdfafc49bbb3152bccc76f788c, State: Running, Role: LEADER
I20260924 21:21:13.722352 11295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } }
I20260924 21:21:13.722801 11278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a8e38fdfafc49bbb3152bccc76f788c. Latest consensus state: current_term: 1 leader_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.722941 11278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:13.723215 11298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260924 21:21:13.723521 11298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260924 21:21:13.720805 11040 master_runner.cc:186] Error getting master registration for 127.20.115.253:35035: OK, instance_id { permanent_uuid: "2e302bf883c4437c82cf24753620849d" instance_seqno: 1790284873509029 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260924 21:21:13.723708 11040 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260924 21:21:13.725610 11232 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [term 1 FOLLOWER]: Refusing update from remote peer 6a8e38fdfafc49bbb3152bccc76f788c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:21:13.725921 11295 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [LEADER]: Connected to new peer: Peer: permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:21:13.727216 11264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a8e38fdfafc49bbb3152bccc76f788c. Latest consensus state: current_term: 1 leader_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.727313 11264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:13.736099 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.721904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44392 (local address 127.20.115.252:39701)
0924 21:21:13.721953 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.721955 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.725708 (+ 3753us) server_negotiation.cc:407] Connection header received
0924 21:21:13.725866 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.725867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.725894 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.725925 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.727477 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.728252 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.729360 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.729659 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.729754 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.731408 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.731410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.731410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.731418 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.733343 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.733734 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.733735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.733736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.733811 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.734784 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.734785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.734786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.734947 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.735070 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.735909 (+ 839us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.736000 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260924 21:21:13.741001 11183 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } attempt: 3
I20260924 21:21:13.744052 11110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.725032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35164 (local address 127.20.115.254:43267)
0924 21:21:13.725078 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.725080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.726679 (+ 1599us) server_negotiation.cc:407] Connection header received
0924 21:21:13.726927 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.726928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.726940 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.726994 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.733769 (+ 6775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.734519 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.735739 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.736055 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.736124 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.741179 (+ 5055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.741181 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.741181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.741189 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.742779 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.743152 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.743153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.743154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.743209 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.743410 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.743411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.743411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.743545 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.743632 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.743860 (+ 228us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.743939 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:21:13.745245 11278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.745399 11278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:13.745893 11264 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.745990 11264 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b [sys.catalog]: This master's current role is: FOLLOWER
Sep 24 21:21:13 dist-test-slave-cxpd krb5kdc[11027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790284873, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.115.253@KRBTEST.COM for kudu/127.20.115.252@KRBTEST.COM
I20260924 21:21:13.750475 11185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.741265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.115.252:39701 (local address 127.0.0.1:44400)
0924 21:21:13.741352 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0924 21:21:13.741364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0924 21:21:13.741402 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0924 21:21:13.741648 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0924 21:21:13.741649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0924 21:21:13.741843 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0924 21:21:13.741887 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.741888 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.742794 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.742795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.743581 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0924 21:21:13.743584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.744060 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.744062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0924 21:21:13.744118 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.745656 (+ 1538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0924 21:21:13.745675 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0924 21:21:13.747105 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0924 21:21:13.749144 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.749145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.749146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.749382 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.749766 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0924 21:21:13.749767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0924 21:21:13.749768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0924 21:21:13.749834 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0924 21:21:13.750148 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0924 21:21:13.750149 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0924 21:21:13.750208 (+ 59us) client_negotiation.cc:769] Sending connection context
0924 21:21:13.750280 (+ 72us) client_negotiation.cc:240] Negotiation successful
0924 21:21:13.750374 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260924 21:21:13.750844 11262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.741476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44400 (local address 127.20.115.252:39701)
0924 21:21:13.741517 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.741518 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.741526 (+ 8us) server_negotiation.cc:407] Connection header received
0924 21:21:13.741547 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.741548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.741558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.741604 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.741974 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.742684 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.743666 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.743955 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.744013 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.747244 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.747245 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.747245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.747267 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.749034 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.749617 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.749618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.749619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.749683 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.749910 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.749911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.749912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.750050 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.750106 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.750680 (+ 574us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.750758 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:21:13.752303 11183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d: Bootstrap starting.
I20260924 21:21:13.752842 11298 catalog_manager.cc:1347] Generated new cluster ID: 57a901fbb7ac41f5a456fc65958a4212
I20260924 21:21:13.752995 11298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260924 21:21:13.753036 11183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d: Neither blocks nor log segments found. Creating new log.
I20260924 21:21:13.753361 11183 log.cc:824] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d: Log is configured to *not* fsync() on all Append() calls
I20260924 21:21:13.754441 11186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:13.725263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41654 (local address 127.20.115.253:35035)
0924 21:21:13.725307 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.725309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.736505 (+ 11196us) server_negotiation.cc:407] Connection header received
0924 21:21:13.736723 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.736724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.736736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.736769 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.737539 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.738245 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.739518 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:13.739831 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:13.739873 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:13.743468 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0924 21:21:13.743470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0924 21:21:13.743471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0924 21:21:13.743478 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0924 21:21:13.745550 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.749483 (+ 3933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.749484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.749484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.749545 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0924 21:21:13.753623 (+ 4078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0924 21:21:13.753624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0924 21:21:13.753625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0924 21:21:13.753756 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0924 21:21:13.753850 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:13.754281 (+ 431us) server_negotiation.cc:299] Negotiation successful
0924 21:21:13.754353 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260924 21:21:13.755401 11183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d: No bootstrap required, opened a new log
I20260924 21:21:13.757515 11183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } }
I20260924 21:21:13.757652 11183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e302bf883c4437c82cf24753620849d, State: Initialized, Role: FOLLOWER
I20260924 21:21:13.757735 11183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } }
I20260924 21:21:13.758111 11183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [sys.catalog]: configured and running, proceeding with master startup.
I20260924 21:21:13.758154 11157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2e302bf883c4437c82cf24753620849d"
I20260924 21:21:13.758276 11157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [term 0 FOLLOWER]: Advancing to term 1
I20260924 21:21:13.759056 11157 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a8e38fdfafc49bbb3152bccc76f788c in term 1.
I20260924 21:21:13.759238 11156 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [term 1 FOLLOWER]: Refusing update from remote peer 6a8e38fdfafc49bbb3152bccc76f788c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260924 21:21:13.759647 11278 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260924 21:21:13.764889 11307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.764997 11307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:13.765061 11307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a8e38fdfafc49bbb3152bccc76f788c. Latest consensus state: current_term: 1 leader_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.765131 11307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [sys.catalog]: This master's current role is: FOLLOWER
I20260924 21:21:13.765173 11307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.765223 11307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d [sys.catalog]: This master's current role is: FOLLOWER
W20260924 21:21:13.765865 11318 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260924 21:21:13.765921 11318 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260924 21:21:13.767148 11320 mvcc.cc:205] Tried to move back new op lower bound from 7333006842777411584 to 7333006842767560704. Current Snapshot: MvccSnapshot[applied={T|T < 7333006842777411584}]
I20260924 21:21:13.767920 11278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6a8e38fdfafc49bbb3152bccc76f788c" member_type: VOTER last_known_addr { host: "127.20.115.254" port: 43267 } } peers { permanent_uuid: "2e302bf883c4437c82cf24753620849d" member_type: VOTER last_known_addr { host: "127.20.115.253" port: 35035 } } peers { permanent_uuid: "12ee677e0f054626bc486dc984435c3b" member_type: VOTER last_known_addr { host: "127.20.115.252" port: 39701 } } }
I20260924 21:21:13.768023 11278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c [sys.catalog]: This master's current role is: LEADER
I20260924 21:21:13.769243 11298 catalog_manager.cc:1370] Generated new certificate authority record
I20260924 21:21:13.769939 11298 catalog_manager.cc:1504] Loading token signing keys...
I20260924 21:21:13.785122 11298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a8e38fdfafc49bbb3152bccc76f788c: Generated new TSK 0
I20260924 21:21:13.785318 11298 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260924 21:21:13.822496 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.820857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.20.115.253:35035)
0924 21:21:13.820900 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.820902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.821690 (+ 788us) server_negotiation.cc:407] Connection header received
0924 21:21:13.821908 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.821909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.821922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.821955 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.822428 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41662: BlockingRecv error: recv got EOF from 127.0.0.1:41662 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 21:21:13.823175 11110 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.820580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35174 (local address 127.20.115.254:43267)
0924 21:21:13.820624 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.820626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.821850 (+ 1224us) server_negotiation.cc:407] Connection header received
0924 21:21:13.822720 (+ 870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.822721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.822732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.822761 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.823111 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35174: BlockingRecv error: recv got EOF from 127.0.0.1:35174 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 21:21:13.823968 11262 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.821401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.20.115.252:39701)
0924 21:21:13.821442 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.821444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.823420 (+ 1976us) server_negotiation.cc:407] Connection header received
0924 21:21:13.823574 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.823575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.823585 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.823615 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.823911 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44414: BlockingRecv error: recv got EOF from 127.0.0.1:44414 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 21:21:13.826323 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.825583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.20.115.253:35035)
0924 21:21:13.825619 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.825620 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.825799 (+ 179us) server_negotiation.cc:407] Connection header received
0924 21:21:13.825905 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.825906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.825923 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.825954 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.826264 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41670: BlockingRecv error: recv got EOF from 127.0.0.1:41670 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260924 21:21:13.829344 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.828543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41684 (local address 127.20.115.253:35035)
0924 21:21:13.828577 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.828579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.828850 (+ 271us) server_negotiation.cc:407] Connection header received
0924 21:21:13.828946 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.828947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.828957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.828984 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.829298 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41684: BlockingRecv error: recv got EOF from 127.0.0.1:41684 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260924 21:21:13.833230 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.832460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.20.115.253:35035)
0924 21:21:13.832496 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.832498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.832724 (+ 226us) server_negotiation.cc:407] Connection header received
0924 21:21:13.832854 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.832855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.832868 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.832896 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.833189 (+ 293us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41688: BlockingRecv error: recv got EOF from 127.0.0.1:41688 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260924 21:21:13.839069 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.838237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41694 (local address 127.20.115.253:35035)
0924 21:21:13.838294 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.838296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.838560 (+ 264us) server_negotiation.cc:407] Connection header received
0924 21:21:13.838661 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.838661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.838672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.838700 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.839018 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41694: BlockingRecv error: recv got EOF from 127.0.0.1:41694 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 21:21:13.849464 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.848369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.20.115.253:35035)
0924 21:21:13.848413 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.848415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.848730 (+ 315us) server_negotiation.cc:407] Connection header received
0924 21:21:13.848960 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.848962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.848975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.849004 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.849404 (+ 400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41706: BlockingRecv error: recv got EOF from 127.0.0.1:41706 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260924 21:21:13.867638 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.866728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41720 (local address 127.20.115.253:35035)
0924 21:21:13.866772 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.866774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.867054 (+ 280us) server_negotiation.cc:407] Connection header received
0924 21:21:13.867156 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.867157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.867168 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.867197 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.867574 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41720: BlockingRecv error: recv got EOF from 127.0.0.1:41720 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260924 21:21:13.901705 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.900912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41734 (local address 127.20.115.253:35035)
0924 21:21:13.900962 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.900965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.901113 (+ 148us) server_negotiation.cc:407] Connection header received
0924 21:21:13.901189 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.901190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.901202 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.901295 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.901637 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41734: BlockingRecv error: recv got EOF from 127.0.0.1:41734 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260924 21:21:13.967501 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:13.966823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41736 (local address 127.20.115.253:35035)
0924 21:21:13.966905 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:13.966907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:13.966966 (+ 59us) server_negotiation.cc:407] Connection header received
0924 21:21:13.967018 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:13.967019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:13.967029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:13.967083 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:13.967443 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41736: BlockingRecv error: recv got EOF from 127.0.0.1:41736 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260924 21:21:14.097591 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:14.096731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41750 (local address 127.20.115.253:35035)
0924 21:21:14.096846 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:14.096848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:14.096962 (+ 114us) server_negotiation.cc:407] Connection header received
0924 21:21:14.097008 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:14.097009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:14.097019 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:14.097148 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:14.097510 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41750: BlockingRecv error: recv got EOF from 127.0.0.1:41750 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260924 21:21:14.355310 11186 negotiation.cc:337] Failed RPC negotiation. Trace:
0924 21:21:14.354740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41756 (local address 127.20.115.253:35035)
0924 21:21:14.354812 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:14.354814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:14.354825 (+ 11us) server_negotiation.cc:407] Connection header received
0924 21:21:14.354867 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:14.354868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:14.354878 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:14.354927 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0924 21:21:14.355262 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41756: BlockingRecv error: recv got EOF from 127.0.0.1:41756 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260924 21:21:14.699492 11275 catalog_manager.cc:1259] Loaded cluster ID: 57a901fbb7ac41f5a456fc65958a4212
I20260924 21:21:14.699573 11275 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b: loading cluster ID for follower catalog manager: success
I20260924 21:21:14.700333 11275 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b: acquiring CA information for follower catalog manager: success
I20260924 21:21:14.700585 11275 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 12ee677e0f054626bc486dc984435c3b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:21:14.766402 11318 catalog_manager.cc:1259] Loaded cluster ID: 57a901fbb7ac41f5a456fc65958a4212
I20260924 21:21:14.766480 11318 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d: loading cluster ID for follower catalog manager: success
I20260924 21:21:14.767230 11318 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d: acquiring CA information for follower catalog manager: success
I20260924 21:21:14.767479 11318 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2e302bf883c4437c82cf24753620849d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260924 21:21:14.871043 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:14.868552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.20.115.253:35035)
0924 21:21:14.868652 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:14.868654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:14.868696 (+ 42us) server_negotiation.cc:407] Connection header received
0924 21:21:14.868722 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:14.868722 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:14.868771 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:14.868888 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:14.869060 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.869569 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.870340 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.870539 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.870612 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:14.870756 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:14.870866 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:14.870926 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:14.870955 (+ 29us) server_negotiation.cc:299] Negotiation successful
0924 21:21:14.870975 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":50,"threads_started":1}
I20260924 21:21:14.875304 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:14.872896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44430 (local address 127.20.115.252:39701)
0924 21:21:14.872999 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:14.873001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:14.873080 (+ 79us) server_negotiation.cc:407] Connection header received
0924 21:21:14.873153 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:14.873153 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:14.873187 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:14.873270 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:14.873349 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.873815 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.874508 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.874737 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.874800 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:14.874930 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:14.875073 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:14.875135 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:14.875218 (+ 83us) server_negotiation.cc:299] Negotiation successful
0924 21:21:14.875238 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":52,"threads_started":1}
I20260924 21:21:14.880164 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:14.877248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44432 (local address 127.20.115.252:39701)
0924 21:21:14.877296 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:14.877298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:14.877472 (+ 174us) server_negotiation.cc:407] Connection header received
0924 21:21:14.877579 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:14.877580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:14.877591 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:14.877620 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:14.877819 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.878331 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.879381 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.879662 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.879726 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:14.879886 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:14.879918 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:14.879978 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:14.880047 (+ 69us) server_negotiation.cc:299] Negotiation successful
0924 21:21:14.880071 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260924 21:21:14.880447 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:14.877183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41768 (local address 127.20.115.253:35035)
0924 21:21:14.877227 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:14.877229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:14.877451 (+ 222us) server_negotiation.cc:407] Connection header received
0924 21:21:14.877508 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:14.877509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:14.877522 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:14.877578 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:14.877699 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.878540 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.879532 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.879809 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.879878 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:14.880143 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:14.880174 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:14.880201 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:14.880323 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 21:21:14.880350 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260924 21:21:14.880750 11401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0924 21:21:14.877139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35182 (local address 127.20.115.254:43267)
0924 21:21:14.877314 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0924 21:21:14.877317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0924 21:21:14.877433 (+ 116us) server_negotiation.cc:407] Connection header received
0924 21:21:14.877559 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0924 21:21:14.877560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0924 21:21:14.877621 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0924 21:21:14.877702 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0924 21:21:14.877905 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.878732 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.879537 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0924 21:21:14.879858 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0924 21:21:14.879899 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0924 21:21:14.880166 (+ 267us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0924 21:21:14.880288 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0924 21:21:14.880328 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0924 21:21:14.880450 (+ 122us) server_negotiation.cc:299] Negotiation successful
0924 21:21:14.880486 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":49,"threads_started":1}
I20260924 21:21:14.886656 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 11040
I20260924 21:21:14.892236 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 11115
I20260924 21:21:14.897399 20943 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskNZGLg7/build/release/bin/kudu with pid 11190
2026-09-24T21:21:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2897 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5672 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (268033 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST